[BATCH] Scrape your roms on your PC - fastscraper

[elvis56nl]

@paradadf
With some creative thinking i was able to make the script do what I had in mind. I decided I wanted my images in subfolders without suffixes as this makes the roms and media work in both emulation station and attract mode easily.

I ended up making several batchfiles: fastscraper_box2d.bat, fastscraper_box3d.bat, fastscraper_snap.bat, fastscraper_mix3.bat which all downloaded the specified image type into subfolders. After running all 4 batch files I end up with all available images in the romdir sorted into the subfolders box2d, box3d, mix3, snap and video. I feel this is the best way to store roms and media files as it is self contained and can easily be moved from one installation to another.

[paradadf]

@elvis56nl good you found the script useful.
Anyway, currently you can't download every single media type from screenscraper with sselph's scraper, as you wanted.

[elvis56nl]

Messing around with the batch file I came across something strange, when console image type is set to mix3 and on running the script you select all it downloads the 2d cover images. When selecting a specific system when running the script it downloads the mix3 images correctly.

I'm not sure if this is because of something I changed in the script as I can't run any of the fastscraper scripts released in the last 8 months or so unmodified. I always get an error on detecting system language.

[paradadf]

@elvis56nl the script isn't guilty of that... it happens when I scrape too, sometimes.

About the language recognition... you could force it by placing the right parameter in the right place. Otherwise I'd need some logs to see where the problem is.

[Anonpebbles]

Hello, this is a minor issue but I'm currently trying to scrape my dreamcast games, however it doesnt seem to recognise the "dreamcast" folder. I have tried renaming it to dc, segadreamcast and dreamcast but no results. This text file appears in the folder after attempting scraping:
Game,Error,Hash,Extra dreamcast\scraper.exe,hash not found,,

Any help would be greatly appreciated! Thanks fellas

[paradadf]

@anonpebbles the scraper doesn’t scrape the folder but whatever rom file you have inside of it.

[Javier Sandoval]

great exellent tool, thanks alot, i just had a problem with the batch getting stuck as it couldnt recognise the OS lenguage, i deleted that line of code, and after that it worked nice. Ive seen that other dude had a similar problem and thats how i solved mine, so perhaps you might want to look at that part of the code, im using windows 7x64 with sp1 in spanish mexico type. Thanks again.

[paradadf]

@javier-sandoval si fueras tan amable de abrir un issue en github con informacion mas detallada del error, podria echarle un vistazo para intentar evitar eso.

[Javier Sandoval]

@paradadf said in [BATCH] Scrape your roms on your PC - fastscraper:

@javier-sandoval si fueras tan amable de abrir un issue en github con informacion mas detallada del error, podria echarle un vistazo para intentar evitar eso.

claro que si,gracias

[IceLancer]

Hey!
I have constant crash when attempting to scrape
Every single time it will crash on 5 specific games in my mame game list.
Here is data

2017/11/08 00:10:18 arcade srcs:
2017/11/08 00:10:18 console srcs: ss,gdb,ovgdb
2017/11/08 00:10:18 INFO: Checking for new hash.csv.
2017/11/08 00:10:19 INFO: hash.csv "d9eedf6e622a8ab6a1a14486bb90ffb8" up to date.
2017/11/08 00:10:20 INFO: Checking for new OpenVGDB.
2017/11/08 00:10:20 INFO: Upgrading OpenGDB: -> "bc42a41ec8f627815ea1dee9dfcd9514".
2017/11/08 00:10:22 INFO: Upgrade Complete.
2017/11/08 00:10:22 INFO: Starting: test\columns.zip
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x28 pc=0x709c45]

goroutine 65 [running]:
github.com/sselph/scraper/rom.getImage(0x9470c0, 0xc042047d80, 0x0, 0x0, 0xc04293a1e0, 0x16, 0x190, 0x0, 0x0, 0x0)
/home/sselph/go/src/github.com/sselph/scraper/rom/rom.go:366 +0xf5
github.com/sselph/scraper/rom.(*ROM).XML(0xc042e2ba80, 0x9470c0, 0xc042047d80, 0xc042049400, 0x6, 0x8, 0xc04203fb10)
/home/sselph/go/src/github.com/sselph/scraper/rom/rom.go:495 +0x62d
main.worker(0x9470c0, 0xc042047d80, 0xc042940180, 0x6, 0x8, 0xc042049400, 0xc04203fb10, 0xc04301a480, 0xc04301a4e0, 0xc042280a80)
/home/sselph/go/src/github.com/sselph/scraper/scraper.go:140 +0x4e8
created by main.crawlROMs
/home/sselph/go/src/github.com/sselph/scraper/scraper.go:223 +0x66b

Start : 0:10:18.64
Finish : 0:10:23.40
---------------
Duration : 00:00:04,76

There is nothing in the folder except one single game. Columns.zip
And as i said regular scrape without ur batch file will go throu that without error.
Do u know whats causing this?

Ok i went and tested each line separately it seems that -download_marquees="true" is causing the crash.. i cant tell why..

---

ok Sorry it has nothing to do with ur batch, i have posted issue on sselph git

[paradadf]

@icelancer yep, exactly. That's something sselph needs to look into.

[IceLancer]

Add an option to turn off scraper.exe autoupdate. its easy to compile from git with go. And bat keeps bringing old exe instead of my new compiled one. Had to manually cut the code for update.

[paradadf]

@icelancer have you tried increasing scraper.exe version number before compiling? I‘d need to check that. If you can, please open an issue about this on github do I don‘t forget about it, please.

[IceLancer]

@paradadf Should i have? I just compiled from git

BTW , answer me this if u can , thanks.
Is it possible to tell batch when asking for system name , to give him folder name with space ?
I am in process of sorting rom folder to their original names and i am stuck with this bat now.

i tried :
Sega Dreamcast
"Sega Dreamcast"
'Sega Dreamcast'
it keeps finding Sega OR Dreamcast, as not found. But not Sega Dreamcast

[IceLancer]

@icelancer meh not even "all" can see folders with spaces..
for /F "delims=" %f in ('dir /B /A:D') do set system=!system! %f
Lists folders properly:
set system=!system! Sega Dreamcast

Why isnt it finding Sega Dreamcast then?

[IceLancer]

This post is deleted!

[paradadf]

@icelancer EmulationStation doesn‘t have system names containing more than one word (or spaces), so I didn‘t bother taking that case into account. Do you really need it? Why don‘t you use folder names like ES?

About your first question... could you send me your compiled version so I can test it?
Maybe changing == by GEQ in the update check might be enough.

[IceLancer]

@paradadf Sorry but yeah, i am kinda heavy on heaving everything sorted top notch. Making rom folder names to their specs is priority to me. I already spent a week modifing all database to match it.
it also makes all 3 frontend systems working in sync
BUT its not priority for you at all. I found workaround for batch fastscraper by doing dirty hack to it.

here is scraper

https://mega.nz/#!MxQ0HJQa!h6RU6cHjV83F8T7d7wPDOfFJdwo_P8t0EKz-ruVDa4c

it reads :
fastscraper ver. 14.09.2017

[paradadf]

@icelancer I‘ll see what I can do. Hopefully I find some time to test one thing before sselph uploads a newer version
I meant the scrapers version, not fastscrapers version. But don‘t worry... your compiled version should be enough.

[paradadf]

@icelancer there is something weird with your compiled version. When running a batch with the command scraper.exe -version you should get something like v1.4.4 and not an empty string. Moreover, your scraper.exe is over 8 mb while the one sselph offers is only 5,5 mb. I´m not sure where in his code the versioning is being made, but you´ll need at least the same version number as the latest one available on github for the update script to be skipped. Erasing that part of the code is easier (if you really need it) instead of adding a switch.

About your folder names with spaces... I´ll look into it when I have time.