Benchmarking script

[Dragu]

@Substring
On boards where dosbox is available under Recalbox (I dont think benchmark is usefull outside Recalbox), you can already use the tool that I posted for download. Could be interesting to compare PI3, Tinkerboard and XU4 under same Recalbox version. PCPbench

[voljega]

@gaetan Les deux KI et Golden Axe : The Duel

[Gaetan]

@voljega j'ai juste édité mon poste au dessus

[voljega]

@gaetan du coup je suis vraiment surpris pour Golden Axe The Duel qui ne marche pas du tout chez moi...

Est ce que tu peux préciser aussi pour cesvtrois jeux sous quel émulateur ça marche : mame2003, mame2010 ou fba_libretro ?

Et est ce que tu es sur pi, odroid ou pc ?

[Gaetan]

@voljega Je suis sur Pi3,
KI : Mame 0.78
GAD : Mame 2010

[voljega]

@gaetan hmmm et tu sais si tu as un bios en plus pour GA:The Duel ? et nom de la rom ?

[Gaetan]

@voljega Oui, effectivement j'ai ajouté un dossier zip bios dans le dossier des roms.

la rom s'appelle : gaxeduel.zip
Le dossier bios : stvbios.zip

J'ai passé toutes ma soirée de hier à tester une dizaine de version de Golden Axe Duel avec tous les core
Comme dis, ça fonctionne, il faut être patient au démarrage de la rom et ça lague pas mal. Mais eh

[Substring]

Guys english please

@Dragu i dont get the link with dosbox ??

[Gaetan]

@Substring Sorry, back to english

[voljega]

@gaetan OK thanks I will try to test all that and add it to BestArcade if I manage to reproduce it

[Substring]

@gaetan use the webmanager for nicer screenshots

And give your romfile md5 to help voljega

[voljega]

@substring I have the right romsets so I don't need md5, and they can be different according to the type of romset, ie split, non-merged, etc

[Gaetan]

@substring I don't know what is a md5, but I'll be happy to give it

Better like this ? :

@voljega if I can help ?

[Substring]

@gaetan oh yeah
Md5 is a the hash of a file. It's unique for any file

Regarding KI, could you get it to launch on mame2010 ?

[Gaetan]

@substring No, KI 1&2 doesn't launch with mame2010

Here is my gamelist :

I don't know how get rid of the "killer instinct (v1.50) and killer instinct 2 (v1.4) in the gamelist

[Dragu]

@Substring
Hi, it was here : https://forum.recalbox.com/topic/13588/strange-error-in-dosbox/58

[voljega]

@gaetan said in Benchmarking script:

@substring No, KI 1&2 doesn't launch with mame2010
I don't know how get rid of the "killer instinct (v1.50) and killer instinct 2 (v1.4) in the gamelist

This is done by editing your gamelist.xml or configuring your scraper....
Can you also give me the exact rom names for KI & KI2 ?

[Gaetan]

@voljega Here are the names of the roms :

• kinst.zip
• kinst2.zip

Oh yes ? it's something that I can find in the forum ? I couldn't find it.

[Substring]

@dragu now i remember. So a few facts :

• it's dos only
• i'm not even sure it has some text output i could read.

As for now, i prefer focusing on retroarch only because i can test most of systems with the same code

[Substring]

So, here is a first shot :

#!/bin/bash

# read parameters
# $1 is the core
RA_CORE="-L /usr/lib/libretro/$1_libretro.so"

# $2 is the rom
RA_ROM="$2"

# Consider the game should run at 60Hz
FPS=60

# Start the emu
NB_FRAMES=600
RA_OPTS="--verbose --max-frames=${NB_FRAMES}"
RA_CONFIG="--config /recalbox/share/system/configs/retroarch/retroarchcustom.cfg"
RA_OUTPUT_LOG=/tmp/ra.log
rm $RA_OUTPUT_LOG

/usr/bin/time -v /usr/bin/retroarch $RA_OPTS $RA_CORE $RA_CONFIG "$RA_ROM" > $RA_OUTPUT_LOG 2>&1

# Get the emulator output
FRAMES_EXPECTED=`grep "[INFO] Threaded video stats" "$RA_OUTPUT_LOG" | sed "s+.*Frames pushed: \([0-9]*\).*+\1+"`
FRAMES_DROPPED=`grep "[INFO] Threaded video stats" "$RA_OUTPUT_LOG" | sed "s+.*Frames dropped: \([0-9]*\).*+\1+"`
FRAMES_RENDERED=$(($FRAMES_EXPECTED - $FRAMES_DROPPED))
TIME_RUN=`grep "Elapsed (wall clock) time (h:mm:ss or m:ss):" "$RA_OUTPUT_LOG" | sed "s/.*: \([0-9]*\)m \([0-9]*\)\.\([0-9]*\)s/\1 \2\ \3/"`
NB_MINS=`echo $TIME_RUN | cut -d ' ' -f 1`
NB_SECS=`echo $TIME_RUN | cut -d ' ' -f 2`
NB_CENS=`echo $TIME_RUN | cut -d ' ' -f 3`
CPU_LOAD=`grep "Percent of CPU this job got:" "$RA_OUTPUT_LOG" | sed "s/.*: \([0-9]*\)%.*/\1/"`

# Do what we need
# The game should have run for NB_FRAMES / FPS seconds, how long did it really take ?
EXPECTED_TIME=$(($NB_FRAMES / $FPS))
REAL_TIME=$(($NB_MINS * 60 + $NB_SECS))
OVERHEAD=$(($REAL_TIME - $EXPECTED_TIME))

# Looks like RA start emulation at [INFO] Initializing rewind buffer with size: 10 MB
# or [INFO] Loading history file: [/recalbox/share/system/configs/retroarch/content_image_history.lpl].
# And it ends at [INFO] Saved core options file to "/recalbox/share/system/configs/retroarch/cores/retroarch-core-options.cfg"

echo "$FRAMES_EXPECTED - $FRAMES_DROPPED = $FRAMES_RENDERED ($EXPECTED_TIME + $OVERHEAD / $CPU_LOAD)"
echo -n "1st score (100 max):"
python -c "print round(100*$FRAMES_RENDERED/$NB_FRAMES*$NB_FRAMES/$FPS/${REAL_TIME}.${NB_CENS}, 2)"

The final result is a mark out of 100. Shouldn't be above 100. 100 would mean instant load of retroarch + start of the game, which is nearly impossible.

The basic idea : retroarch is run for 600 frames (assume the game is 60Hz). Then compute a few details :

• frames rendered
• frames dropped
• total time elapsed
• CPU %

Made a few tests with killing blade (the PGM logo is already stuttering on a non overclocked pi3)

Pi3 output example:

# bash recalbox-bench.sh fba ../roms/fba_libretro/killbldp.zip    
600 - 0 = 600 (10 + 4 / 88)
1st score (100 max):67.98

C2 output example:

# bash recalbox-bench.sh fba ../roms/fba_libretro/killbld.zip    
600 - 0 = 600 (10 + 2 / 69)
1st score (100 max):81.23

Small explanation on the first result line
600 : number of frames expected
0 : number of frames dropped
10 : expected run time (600 frames at 60 Hz)
2 : run time overhead (means the full length of retroarch took 12s, despite i'm not showing the rest of the value but do have it in the script)
69 : CPU percentage

So from here you can notice:

• the pi3 took something like 14 to 15s for the overall launch of RA + ROM execution + quit RA whereas it took 12 to 13s on the C2
• the PGM logo was consuming 88% on pi3 vs 69% on the C2. And I tell you : the sound doesn't stutter on the C2

Overall, this bench doesn't look that bad for now