[BATCH] Scrape your roms on your PC - fastscraper

[paradadf]

@airart mamedb is down and thus, unscrapable. I've seen that sselph just added a mirror for it and enabled SS as source for mame as well. He just hasn't compiled and released a new version with all those changes yet so I can test and upgrade the batch.

[paradadf]

fastscraper has been upgraded and is fully functional again.
New feature: you can open a folder browser and choose the roms folder of your recalbox directly over the network to scrape from it!
I have set ScreenScraper as the only source because the quality is superior to any other. You can expand or modify it easily if you wish. Enjoy!

[kevinnash]

Hi,

How do we download it on github ?

I don't see any download link there.

[paradadf]

You can select RAW and copy the plain text to a txt file and rename the extension to .bat

[maximus37]

This tool is fantastic!! Nice work!!

One problem though, I cannot scrape NeoGeo roms, I get the following output:

2017/01/10 10:06:56 arcade  srcs: ss
2017/01/10 10:06:56 console srcs:
2017/01/10 10:06:56 INFO: Starting: C:\Users[...]\Downloads\neogeo\3countb.zip
2017/01/10 10:06:56 INFO: Starting: C:\Users[...]\Downloads\neogeo\2020bba.zip
2017/01/10 10:06:56 INFO: Starting: C:\Users[...]\Downloads\neogeo\2020bbh.zip
2017/01/10 10:06:56 INFO: Starting: C:\Users[...]\Downloads\neogeo\2020bb.zip
2017/01/10 10:06:58 ERR: error processing C:\Users[...]\Downloads\neogeo\2020bb.zip: image: unknown format
2017/01/10 10:06:58 INFO: Starting: C:\Users[...]\Downloads\neogeo\2020bb.zip
2017/01/10 10:06:58 ERR: error processing C:\Users\\Downloads\neogeo\2020bbh.zip: image: unknown format
2017/01/10 10:06:58 INFO: Starting: C:\Users[...]\Downloads\neogeo\2020bbh.zip
2017/01/10 10:06:58 ERR: error processing C:\Users[...]\Downloads\neogeo\3countb.zip: image: unknown format
2017/01/10 10:06:58 INFO: Starting: C:\Users[...]\Downloads\neogeo\3countb.zip
2017/01/10 10:06:58 ERR: error processing C:\Users[...]\Downloads\neogeo\2020bba.zip: image: unknown format
2017/01/10 10:06:58 INFO: Starting: C:\Users[...]\Downloads\neogeo\2020bba.zip
2017/01/10 10:06:58 ERR: error processing C:\Users[...]\Downloads\neogeo\2020bb.zip: image: unknown format
2017/01/10 10:06:58 INFO: Starting: C:\Users[...]\Downloads\neogeo\2020bb.zip
2017/01/10 10:06:58 ERR: error processing C:\Users[...]\Downloads\neogeo\2020bbh.zip: image: unknown format
2017/01/10 10:06:58 INFO: Starting: C:\Users[...]\Downloads\neogeo\2020bbh.zip
2017/01/10 10:06:58 ERR: error processing C:\Users[...]\Downloads\neogeo\3countb.zip: image: unknown format
2017/01/10 10:06:58 INFO: Starting: C:\Users[...]\Downloads\neogeo\3countb.zip
2017/01/10 10:06:58 ERR: error processing C:\Users[...]\Downloads\neogeo\2020bba.zip: image: unknown format
2017/01/10 10:06:58 INFO: Starting: C:\Users[...]\Downloads\neogeo\2020bba.zip
2017/01/10 10:06:58 ERR: error processing C:\Users[...]\Downloads\neogeo\2020bb.zip: image: unknown format
2017/01/10 10:06:58 INFO: Starting: C:\Users[...]\Downloads\neogeo\alpham2.zip
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x14 pc=0x408fe7]

goroutine 22 [running]:
panic(0x7e9580, 0x113a0030)
        /usr/local/go/src/runtime/panic.go:500 +0x331
main.crawlROMs.func2(0x11472820, 0x114aa900, 0x0, 0x11472810, 0x114755a0, 0x11475520)
        /home/sselph/go/src/github.com/sselph/scraper/scraper.go:295 +0x527
created by main.crawlROMs
        /home/sselph/go/src/github.com/sselph/scraper/scraper.go:328 +0xbd7

Start    : 10:06:56.62
Finish   : 10:06:58.67
         ---------------
Duration : 00:00:02,05

[acris]

Hello @maximus37
the server for arcade is down.

[maximus37]

@acris Oh, well that explains it

How can I check the status of it? How long has it been down? Is there a site I can keep an eye on?

Thanks!

[paradadf]

@maximus37 @acris The problem with NEOGEO right now (using SS as source) is explained here: https://github.com/sselph/scraper/issues/128
So most likely sselph will need to compile a new version with the corrections to allow us to scrape it.

[maximus37]

@paradadf Thanks for the link!

[maximus37]

@paradadf Hi, I noticed the issue was fixed? Do I simply download the latest version of fastscraper? Do I need to include any options for the scraping?

[paradadf]

@maximus37 it wasn't a problem from sselph's scraper, but from the default used database (SS). So the version you have should work. But it is always better to use the latest

About the options, on top of the batch you can modify the different flags to your taste, change sources, type of images, etc. So is up to you which configuration you prefer.

[airart]

I try to download or copy fastscraper.bat new version but server is down

[kevinnash]

Github is OK on my side : https://raw.githubusercontent.com/paradadf/recaltools/master/fastscraper/fastscraper.bat

[airart]

@kevinnash is new version ? is not working .... i'm so sad

[paradadf]

@airart it is working... you need scraper v.1.2.0 (latest version).

[airart]

@paradadf said in [BATCH] Scrape your roms on your PC - fastscraper:

scraper v.1.2.0

Can U give me a link please , i've already copy raw fastscraper

[paradadf]

@airart just delete the scraper.exe you have and launch the batch.

[airart]

@paradadf said in [BATCH] Scrape your roms on your PC - fastscraper:

@airart just delete the scraper.exe you have and launch the batch.

Err processing
....xml syntax error on line 1

[paradadf]

@airart open the batch file, you probably have something else inside as the actual code.

[airart]

@paradadf said in [BATCH] Scrape your roms on your PC - fastscraper:

@airart open the batch file, you probably have something else inside as the actual code.
maybe ..... i have copied raw in existing .bat overwryte all , can you give me a clean .bat ?