Input Lag: Vergleich zwischen Recalbox und SNES Classic Mini

[MarcDK]

Hi,

ich habe nirgendwo einen echten Vergleich zwischen den beiden gefunden. Um es kurz zu machen: Der Unterschied ist leider sehr groß: https://marc.tv/input-lag-nintendo-mini-vs-recalbox-auf-dem-raspberry-pi/

[joinski]

Moin Marc,

hier gibt es einen sehr interessanten Vergleichstest was Input Lag bei SNES Emulation angeht zwischen dem SNES Classic Mini, einem Pi3 mit RetroPie und einem High-End PC mit RetroArch:

https://forums.libretro.com/t/an-input-lag-investigation/4407/583

Wie man sieht, liegt der Pi3 mit RetroPie (wenn man die Settings optimiert) gleich auf mit dem SNES Classic Mini.

Mit Recalbox sollte sich also theoretisch ein ähnliches Ergebnis erzielen lassen.
Sobald ich Zeit hab, muss ich mal schauen, ob man bei Recalbox im SNES-Emulator auch irgendwo diese Settings finden und entsprechend einstellen kann:
video_threaded=false
video_driver=dispmanx
video_max_swapchain_images=2

Die letzte Einstellung (video_max_swapchain_images=2) soll allerdings viel CPU-Leistung benötigen und somit die Performance verschlechtern, was eventuell dazu führen kann, dass bestimmte SNES-Spiele auf dem Pi3 nicht mehr flüssig laufen.
Aber auch wenn man nur die ersten beiden Einstellungen verwendet, sollte der Input Lag wesentlich verringert sein.

Viele Grüße
John

[Substring]

I'd have very much fun debating this in english ...

[MarcDK]

@substring Sure, no problem. I love my recalbox and would gladly learn how to save some ms. But currently the situation is:

152ms with PocketSNES on Recalbox (*)
96ms with SNES Mini
71ms Real SNES

All 3 were loaded with Super Mario World.

And 152 ms is the lowest I am able to get. All measurements where performed on the same setup excluding the real SNES. That was connected to a CRT Monitor.
I tried each and every trick to reduce input lag on recalbox. But with no luck.

*) Fastest Emulator on Recalbox according to this article: https://marc.tv/recalbox-input-lag-bei-den-emulatoren/

[MarcDK]

@joinski The link you posted used open source emulators on SNES Mini Hardware. I measured the SNES Minis official emulator.The console was not altered or hacked in any way.

Further down in the text the guy writes: "We can see that the SNES Mini with it’s default emulator (Canoe) is pretty fast. A real SNES on a CRT would achieve ~3.3 frames in our test case and the SNES Mini achieves ~4.6 frames if we remove the Samsung TV’s input lag. That’s just ~1.3 frames (~22 ms) behind the real thing. That’s pretty awesome and a job well done by Nintendo, especially given the low computational performance of the Mini’s hardware."

Thats basically the same result I got.

[joinski]

@marcdk

In the link that i posted, the user "Brunnis" has tested the SNES Classic Mini with the default emulator (Canoe) and also with RetroArch.

@Substring

Do you know, if there are settings in Recalbox SNES Emulator for "video_threaded" and "video_driver" and where to find them?
I'm at work right now, so I can not check it myself.

[Substring]

@MarcDK i'd like you to make some critics about your own measures, and describe where you lack precision in your tests. Because, to be honnest, i'm afraid you didn't do with the required knowledge for such testing protocol.

You should at least give an error percentage on your measures.

[Substring]

@joinski these settings are pure RA, you can toggle them in RA of course, or manually det them in the config files

[MarcDK]

@Substring I am curious about your comments. I measured every system several times. I used the same rom/game and the same TV (excluding the real snes).

I don't claim to be exact about every millisecond but the big picture is correct. I had help from several other people for the test setup and they checked the results. See thank you section at the end of this article: https://marc.tv/recalbox-input-lag-bei-den-emulatoren/

So please tell me how to significantly reduce the input lag on recalbox with a raspberry pi. I will measure it and extend the article right away.

[Substring]

@marcdk i can't tell how to reduce input lag. I'm not sayng your measures are right or wrong. I'm talking about the proces itself.

So tell me:

• how fast was your cam recording ?
• how can you determine when the button is i deed pressed ?
• was the game 50hz or 60hz ? Was vsync on ?
• how do you measure emulation speed ?
• how do you measure your screen lag ?
• how donyou measure the video signal composition time ?
• how sure are you a out the OS speed by itself ? Its responsiveness to a user input ?

Now that you have a small idea of what needs to be "scientifically measured and criticized", how do you calculate your error percentage ? How many frames does it represent ?

Why haven't you measured the lag on a CRT screen for example ? Do you think it would be worth ? If yes, why ? Give me numbers, formulas, proofs, evidences, not just results.

That's the scientific way of doing things, which is the only one worth when trying to get a real picture of something.

[MarcDK]

@Substring Please translate this article with google translate.

https://translate.google.com/translate?sl=de&tl=en&js=y&prev=_t&hl=de&ie=UTF-8&u=https%3A%2F%2Fmarc.tv%2Frecalbox-input-lag%2F&edit-text=

Every question you have is is answered there.

[Substring]

@marcdk your very first formula is right, but you already have a wrong result ...
If 480 frames take 1s, then 171 represents 171*480/1000 = 82.08 frames ... And you said 84 frames. That's already a 2.5% error right from the start.

But the funniest is that you record at 240 fps and finally decide that it's 480fps because ... Because final cut told you. You don't even know. So maybe the lag is just half of what you said : 171*240/1000=41.04 frames

But these frames are your phone's frames. How many snes frames does that mean ? But hey, is your game is running at 60 or 50Hz ? You're not saying.

So at 60Hz, 171ms means 171*60/1000, so 10.26 fps, which turns to 11fps ... For 50Hz, that's 8.55fps, so 9.
The real snes seems to have 3.6fps input lag, we're 3 times bigger.

When is the button really active when you're recording ? We don't know. Why not just record the frame value on RA at the same time ? You'd have some figures to get i deep

And your screen adds 1.5 frame of input lag ...

How often have you made the experiment to determine the statistic parameters regarding the error measurement ?

How can i trust you when you say that wireless pads have almost no input lag ?

This is not serious at all, despite the effort you put in it. I feel like you're tricking people who have no background knowledge regarding what measuting all this means.

[MarcDK]

@substring said in Input Lag: Vergleich zwischen Recalbox und SNES Classic Mini:

But the funniest is that you record at 240 fps and finally decide that it's 480fps because ... Because final cut told you. You don't even know. So maybe the lag is just half of what you said : 171*240/1000=41.04 frames

"Because final cut told you". Wrong. I filmed a clock to verify my assumtion. That's how I know that 480 frames in Final Cut are on second. It does not matter if it runs with 60 or 50hz if you know how long a seconds is since I did not count frames of the game but the frames in the video to get the ms. I know that many other ppl tell you the frame delay of the game but this is not comparable over all different systems.

As I wrote in the article I filmed the joypad from the side to see if the button was pressed down. With 240 real frames this is accurate enough if you make many test runs. I compared wireless with wired pads in the results table. So I know how much the delay is. I did this is in the recalbox menu which is a very low input lag btw.

As I told you: I don't claim to have accurate results. But I the tendency that a SNES Mini, Wii U and Raspberry Pi with Recalbox have different and reproducable types of input lag which can be brought in an order from "low" (SNES Mini) to "high" (Recalbox on RPI3) is verified. My TV (KS7090, According to NEOGaf one of the best gaming tvs) has a very low input lag with 21ms (see article). My guess is that many other people will experience a much higher input lag with their TVs.

So, I really believe that this test setup show the difference between many different solutions to play emulated games. And show at least a comparable value between the systems. This will vary between different TVs of course. But I used the same TV with the same HDMI Input in GameMode,.

[Substring]

@marcdk A TV is not a good idea to measure input lag as it's much slower than a gamer monitor.

240fps or 480fps is afterall not important, because we just want to measure the time elapsed. Your twice more precise at 480fps of course. I still say that you have no idea when the button was pressed, and its a source of error. That's why retroarch had a great idea to light a LED when the contact is done. That way, you know when the button is pressed.

Having the frames from RA is also something interesting because you'd get the frame at which the LED turned on. At this point, on a fast monitor that reaches 2ms, you know your statistical error when you press the button is of 1 frame maximum. Mario will jump on a precise frame. If it's 4 frames later, you can be sure that it's not more than 5 frames. That's a 25% error. Which you can probably reduce by repeating the experiment many times. that's how you should have proceeded.

Now I also remember that the tests on RA were made using KMS, which we don't have (yet) on the Pi. The pi does have a kms version of the GPU access, and it's said to be (much ?) faster. But retroarch is also compiled with dispmanx renderer, which avoids using SDL2 and is said to be "faster" (rumor says it's 1 frame faster, i have no proof of that).

One other test would be to simply measure the "real" input lag of the OS itself bu running evtest. And here I expect you find differences between USB and BT pads. If not, I'll remind any user who complains that BT lags that you proved it doesn't

Final word : the pi is not a wuick machine made for real time processing. But lakka patched their kernel as a real time system, results for them could be interesting, and I'd say their input lag is better than Recalbox.

Your tests do lack your own criticism where you should first have described what composes the lag in itself, where are measure errors, how you'd evaluate them. You can't just say "Guyz, I tell you, Recalbox Snes9x has a 117ms input lag whatever you do. You're all screwed. And that's why i miss all my jumps in Super Mario".

[joinski]

@Substring

It is not about perfect measurements.
It is about comparison.
In comparison (within the same conditions), SNES emulation on a Pi3 with Recalbox has more input lag, than SNES Classic Mini (Canoe).
RetroArch on Pi3 with certain settings can reach the same low input lag level as SNES Classic Mini (see the link i posted above).

[Substring]

@joinski said in Input Lag: Vergleich zwischen Recalbox und SNES Classic Mini:

It is not about perfect measurements.

That's the whole point of it ! How unperfect it is ! What is the error percentage in all that ? at 10 frames lag input, 1 frame is a 10% error. At 4 fps input lag, 1 frame is 25% of error. Si yes, it is unperfect, but say how much it is. And then the article defintely gives a real proof of seriousness.

[joinski]

@substring said in Input Lag: Vergleich zwischen Recalbox und SNES Classic Mini:

@joinski said in Input Lag: Vergleich zwischen Recalbox und SNES Classic Mini:

It is not about perfect measurements.

That's the whole point of it ! How unperfect it is ! What is the error percentage in all that ? at 10 frames lag input, 1 frame is a 10% error. At 4 fps input lag, 1 frame is 25% of error. Si yes, it is unperfect, but say how much it is. And then the article defintely gives a real proof of seriousness.

No, that is not the whole point of it!
It does not matter how much the error percentage is.

Because, I said it already... it is about comparison.
The point is, that the SNES emulator in Recalbox with default settings has a higher input lag, than it could possibly have, when settings would be optimised.

I just tested with my Pi3 and Recalbox 18.02.09 with following settings:
RetroArch Main Menu (Hotkey + B)

Settings
Driver
Video Driver - dispmanx

Settings
Video
Threaded Video - Off

And i think it feels a bit better (less input lag) than with default settings.
@MarcDK could you test it?

[voljega]

@joinski are you at least comparing snes mini and recalbox with the same kind of pad in both cases ? wireless or not ? because if not and you're using the original gamepad on the snes mini and wireless one in recalbox it doesn't mean anything.

also if you're using internal bt on the pi for your test, it's pretty s**tty

[MarcDK]

The only thing my articles tell the reader is that this devices play Super Mario World with an input lag in ascending order: (from best to worst) on my television (gaming tv with low input lag) with the best possible gamepad setup (wired over wireless if possible)

• SNES Classic Mini
• Wii U Virtual Console
• Recalbox (PocketSNES)
• Recalbox (Snes9x Next)

The SNES Classic Mini has according to my measments by far (!) the lowest input latency. Right now it is not possible to achieve a low input lag with a raspberry pi with the current state of emulation. Many ppl tried but the result is always the same: With the regular emulators you need a powerful pc.

This does not lower the achievement of the Raspberry Pi or Libreto. It's free! But it has trade offs. I believe I am not the only person who felt the input lag right from the start on the raspberry pi (Batocera, Recalbox ,Retropie with and without optimizations). If you play with the SNES Mini in comparision there is no doubt that it responds much faster. Even without my numbers.

[joinski]

@voljega

Ok, you are right. The comparison should have been done with the same gamepad. I do not know how much input lag difference is between the SNES Classic Mini wired gamepad and a wireless Xbox360 or PS3 Controller.

Also i noticed too, that the internal bluetooth of Pi3 is not as good as certain usb bluetooth dongles.

But at least the comparison test of "Brunni" from the RetroArch forum is very enlightening. Did you take a look?
He found out, that with certain settings ("video driver" = dispmanx and "threaded video" = off) you can lower the input lag to a level near to SNES Classic Mini.
@MarcDK
Could you please test Recalbox with your method with the settings i mentioned?

In the libretro-forum i found another interesting thread with a guide to reduce input lag:
https://forums.libretro.com/t/definitive-guide-to-reducing-input-lag-to-the-minimum-possible/3028
Maybe some of the settings mentioned there are also possible to set in Recalbox and will reduce input lag a bit more?