[BATCH] Scrape your roms on your PC - fastscraper

[paradadf]

The fastscraper is an useful batch script to be used with sselph scraper on Windows. With it, you can scrape many systems (some listed here) directly from your Windows PC (really fast!) according to the format needed in recalbox 4.
You just need to put fastscraper.bat and scraper.exe on the root folder of your roms.

And execute fastscraper.bat. For multiple systems you can just type a space between the names or type all to scan all folders, and you can select if you want to update a gamelist or generate a new one. Fast and simple!

Finally you would need to transfer the downloaded_images folder and gamelist.xml to your SD Card.

You must stop EmulationStation before copying the gamelist.xml with the command: /etc/init.d/S31emulationstation stop and start it again after the copy is finished with /etc/init.d/S31emulationstation start.

That's all!

Download the latest version here!

Update 08.01.2017:
You can open a folder browser and navigate directly to the network share of your recalbox and scrape from it! This is, of course, slower than doing it on your computer, it can be useful though.

Update 10.01.2017:
macOS and Linux version available here.

Update 27.01.2017:
Automatic download/update of scraper.exe

Update 25.04.2017:
Mix images for consoles arrived and became the default.

[nickdj]

Works great.  I've been playing with various tools (including the batch files I created many moons ago) and like the simplicity of this one.

[voljega]

Looks simple enough ! A few questions : - what is the size of downloaded images ? - what criteria does it use to find a game ? hash code of the file ? full name of the rom file ? partial full name when no match is found for full name ?

[paradadf]

@Voljega If you open the batch file with a text editor and go to the main code section you can see (and change if you need to) all parameters. -max_width=375 defines the image size. All images are resized (proportionally) to 375 pixels width. I know that sselph scraper uses its own hash database based on thegamesdb.net as default (to be found here (hash.csv)) and I enabled the use of the OpenVGDB if the hash isn't in hash.csv by setting the flag to -use_ovgdb=true, so that more games are found. How exactly the hashing works, I can't say, sorry.  For the tests I did, if you change the name of the rom, the scraper still finds a match on the database. For mame, fba, fba_libretro and neogeo, mamedb is always used.

[sselph]

Sorry for the late response, I only periodically look for mentions of the scraper. The scraper uses hashing for most systems and attempts to do a sha1 hash of the original rom data. To do this, it pa**es and/or strips out additional headers and flips bits around to get a consistent encoding.

[screech]

or the Universal_XML_Scraper which scrape from Windows but only in french ?

Just for information, that's not true ^^ Universal XML Scraper scrape also in english if you choose english langage (and it's automatique now ^^)

[paradadf]

@Screech Sorry for having written that if it isn't true... the last time I tried, everything scraped was in french. I don't know if you changed it in the meantime As there are always some improvements on sselph's side, I want to share with you a new version of fastscraper. Mainly and the most important, now you have the option to choose between scraping NeoGeo in mame mode (using mamedb.com as a source) or using it in non-mame mode (thegamesdb.net). That feature was implemented on v.1.1.4.

Every time NeoGeo is selected, you get the option to choose between those two modes. I hope you find it useful!

Download fastscraper ver. 01.07.2016

[screech]

@paradadf No problemo ^^ it's pretty new (the 'automatic english setup'. You can do it before the new version, but you have to edit the config file manually. Now you can just change your language to change the scraped info )

[paradadf]

As ScreenScraper.fr has been newly added as a source in sselph's scraper, I updated the batch to be able to easily use it. The support for SS is given starting with the scraper's version v1.1.10. Currently only consoles are supported, but you'll probably still have some problems with cd based consoles and/or computers. Arcade (mame, fba, fba_libretro) is with this release still not supported either. mamedb and tgdb are of course working as before.

Starting with this version of the batch, I created on top of the file (just open it with any text editor) a section where you can change parameters such as:

• Change the parameters "b,s" as you prefer.

Comma separated order to prefer images, s=snapshot, b=boxart, f=fanart, a=banner, l=logo, 3b=3D boxart. (default "b")
		set consoleIMG=-console_img="b,s"

• Change between jpg or png image format

jpg or png, the format to write the images. (default "jpg")
		set imgFormat=-img_format="jpg"

• Choose the data sources (if you activate the three of them, currently the hierarchy is gdb > ss > ovgdb)

Use the hash.csv and theGamesDB metadata. (default true)
		set useGDB=-use_gdb=false

Use the OpenVGDB if the hash isn´t in hash.csv.
	       set useOVGDB=-use_ovgdb=false

SS is the only one I activated by default. Just change false for true as you prefer.

Use the ScreenScraper.fr as a datasource.
		set useSS=-use_ss=true

• Chose the language (mainly english and french data currently) and region for the data/media scraped

The order to choose for language if there is more than one for a value. (en, fr, es, de, pt) (default "en")
		set langSS=-lang="en,es,pt,de,fr"

The order to choose for region if there is more than one for a value. (us, eu, jp, fr, xx) (default "us,eu,jp,fr,xx")
		set regionSS=-region="us,eu,jp,fr,xx"

Download fastscraper ver. 04.10.2016

[Substring]

OMG, it's not autoit ! Hey @screech i'm sure this runs faster than UXS

[screech]

@Subs sure it run faster (and on Linux and MAC ^^) but no MIX template (for the moment ^^)

[Substring]

@screech it's just a matter fo time

[screech]

@Subs I know Sselph make some try on MIX

[Substring]

@screech hahaha fear sselph, for your scraper will soon be overwhelmed

[screech]

@Subs Never ^^ I use AUTOIT !! and it's the FUTUR !!! (And wait for the V2 you will see the great power of Autoit )

(Sorry paradadf, we are trolling on your post :S )

[paradadf]

@screech no prob

[The D]

Wow! This thing is fast and powerful. I'm glad I stumbled upon it!

One question- is there a way to prevent the addition of the regions to the display name?

[paradadf]

@The-D try editing this to:
-use_nointro_name=false

[paradadf]

Little update:

Now you can type all to scrape all the folders at once.

I added some extra extension so that, for example, ScummVM can now be scraped, assuming you followed this instructions.

Please, don't forget to contribute to ScreenScraper and make the database bigger!
PC DOS, PC Win3.xx, PC Win9X and PC Windows are newly added systems, so you can now scrape Cave Story and Doom games. (Thinking about doxbox and future ports ;))

Download fastscraper ver. 24.10.2016

[Gonzakpo]

Hi. Thanks for the script. I've just found a little error on it. If I change the parameter where downloaded images are stored, it just changes the value in the gamelist xml. It doesn't change the actual place where they are downloaded (the command is hardcoded to download always on downloaded_images).

By the way, two days ago everything was working great. And now most of my roms are giving me "hash not found" although I verified the hashes and they have not changed. Well known games like 1942 are giving me "hash not found". Is something wrong with the scraper.exe ? I'm using the latest one.