Raspberry Pi 4

For information, Recalbox IS NOT compatible with Raspberry Pi 4 yet.
Pour information, Recalbox N'EST PAS encore compatible avec le Raspberry Pi 4.

The Recalbox Team.

[BATCH] Scrape your roms on your PC - fastscraper



  • @paradadf ok, the 2 items created go in the roms folder. OK, thank you very much.



  • @paradadf I redownloaded and same results. Guess im SOL, thanks for your help anyway.



  • Im getting "too many connections" error the first time I use the FastScrapper.
    I dont know what to do...... any suggest?
    Thanks!!

    Scraping atari2600 in progress. Please wait...
    
    2016/12/06 03:55:01 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping atari7800 in progress. Please wait...
    
    2016/12/06 03:55:01 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping atarist in progress. Please wait...
    
    2016/12/06 03:55:02 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping cavestory in progress. Please wait...
    
    2016/12/06 03:55:02 INFO: Checking for new hash.csv.
    2016/12/06 03:55:02 INFO: hash.csv "e2315a44c3d4c5d200477a109e079f29" up to date.
    
    
    Scraping fba in progress. Please wait...
    
    2016/12/06 03:55:02 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping fba_libretro in progress. Please wait...
    
    2016/12/06 03:55:03 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping fds in progress. Please wait...
    
    2016/12/06 03:55:03 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping gamegear in progress. Please wait...
    
    2016/12/06 03:55:03 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping gb in progress. Please wait...
    
    2016/12/06 03:55:04 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping gba in progress. Please wait...
    
    2016/12/06 03:55:04 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping gbc in progress. Please wait...
    
    2016/12/06 03:55:05 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping gw in progress. Please wait...
    
    2016/12/06 03:55:05 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping lutro in progress. Please wait...
    
    2016/12/06 03:55:05 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping lynx in progress. Please wait...
    
    2016/12/06 03:55:06 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping mame in progress. Please wait...
    
    2016/12/06 03:55:06 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping mastersystem in progress. Please wait...
    
    2016/12/06 03:55:06 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping megadrive in progress. Please wait...
    
    2016/12/06 03:55:07 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping moonlight in progress. Please wait...
    
    2016/12/06 03:55:07 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping msx in progress. Please wait...
    
    2016/12/06 03:55:07 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping msx1 in progress. Please wait...
    
    2016/12/06 03:55:08 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping msx2 in progress. Please wait...
    
    2016/12/06 03:55:08 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping n64 in progress. Please wait...
    
    2016/12/06 03:55:09 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping neogeo in progress. Please wait...
    
    2016/12/06 03:55:09 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping nes in progress. Please wait...
    
    2016/12/06 03:55:09 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping ngp in progress. Please wait...
    
    2016/12/06 03:55:10 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping ngpc in progress. Please wait...
    
    2016/12/06 03:55:10 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping o2em in progress. Please wait...
    
    2016/12/06 03:55:10 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping pcengine in progress. Please wait...
    
    2016/12/06 03:55:11 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping pcenginecd in progress. Please wait...
    
    2016/12/06 03:55:11 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping prboom in progress. Please wait...
    
    2016/12/06 03:55:11 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping psx in progress. Please wait...
    
    2016/12/06 03:55:12 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping scummvm in progress. Please wait...
    
    2016/12/06 03:55:12 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping sega32x in progress. Please wait...
    
    2016/12/06 03:55:13 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping segacd in progress. Please wait...
    
    2016/12/06 03:55:13 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping sg1000 in progress. Please wait...
    
    2016/12/06 03:55:13 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping snes in progress. Please wait...
    
    2016/12/06 03:55:14 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping supergrafx in progress. Please wait...
    
    2016/12/06 03:55:14 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping vectrex in progress. Please wait...
    
    2016/12/06 03:55:14 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping virtualboy in progress. Please wait...
    
    2016/12/06 03:55:15 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping wswan in progress. Please wait...
    
    2016/12/06 03:55:15 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping wswanc in progress. Please wait...
    
    2016/12/06 03:55:15 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping zx81 in progress. Please wait...
    
    2016/12/06 03:55:16 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    
    Scraping zxspectrum in progress. Please wait...
    
    2016/12/06 03:55:16 INFO: Checking for new hash.csv.
    got 429 Too Many Requests response
    
    Start    :  3:55:00,90
    Finish   :  3:55:16,88
             ---------------
    Duration : 00:00:15,98
    
    D:\0_COPIA_SEG_RECALBOX\recalbox\roms>
    

  • Staff
    Team
    Moderator
    Translator

    I'd say you are having too many requests response XD
    That depends on the data source. Have you increased the workers? Probably your IP is doing too many requests in a short period of time and the API is blocking you, I don't know.

    Fastscraper ist just a sort-of interface for sselph's scraper. I don't know exactly what every error displayed by it means, sorry.



  • @paradadf

    It Only happens at night. At mid-day is perfect, so I suppose is a server limit for Ip range.


    Creo que hablas Español, asi que tambien te lo dejo escrito en español :P
    Creo que solo ocurre de noche, a medio dia funciona perfecto. creo que es una limitacion que esta aplicando el servidor a un rango de ips en concreto (yo estoy con movistar)



  • @paradadf
    I'm pretty NooB at this. How do I run commands from the RP3 in order to stop the emulation station before copying? I'm using a Windows PC.


  • Staff
    Team
    Moderator
    Translator

    @pyrostic i strongly recommend you to read the wiki/manuals, as all the basic (and some advanced) festures and tricks are explained there.
    SSH:
    https://github.com/recalbox/recalbox-os/wiki/Root-access-on-terminal-(EN)
    WinSCP:
    https://github.com/recalbox/recalbox-os/wiki/Network-access-with-WinSCP-(EN)



  • Ok. I got the scraper to scrape my ROM files and create the xml file and images folder. When I put my SD card in and open the main folder, all I have is a boot folder of around 50 MB (of a 32 GB card) that doesn't show any of my ROM files to be able to place the images folder and xml file in there. When I put the SD card into my raspberry pi zero and load up Retropie, all my ROMS are on there and I'm able to play them, but it doesn't show any of the image files or game data if I put the folder and xml onto the boot area I'm getting when I plug it into my PC.

    What am I doing wrong? Are the ROMS on a separate partition, or something?


  • Staff
    Team
    Moderator
    Translator

    @Keladir first of all, you are on the recalbox forum. for retropie specific stuff, you should use its forum.

    Now about your "issue"... Windows PCs can't read the extra partitions, so you need special software to access them. And as clear as I was on my instructions on the first post, you need to place both, gamelist.xml and downloaded_images folder next to your roms. So I just don't get why would you place it anywhere else. I doesn't make any sense.



  • @paradadf I opened it up on Retropie's file explorer to poke around, and when I imported my ROMS via USB thumb drive, it copied over the images folder and xml file into the same directory as all the roms, "next to" them as you say, yet they're still not showing up when I load into a console's game list.

    Your initial post doesn't say where on the SD card to transfer the XML and images folder to.


  • Staff
    Team
    Moderator
    Translator

    @Keladir again, this is the recalbox forum, if you have problems transfering the files, make your question on the retropie forum.

    Your initial post doesn't say where on the SD card to transfer the XML and images folder to.

    Next to your roms is a unique location. Wherever the correct place were you copied your roms for retropie, that's the place! I don't know how retropie works, but my initial post also states that you must kill emulationstation before trying to edit/copy/transfer/touch the gamelist.xml. And that is valid for emulationstation in general.

    Sorry but either you don't want to understand, or you are not paying attention to what you read.



  • hi, I copied the roms of my PC, but I get this error
    alt text

    I apologize for my English, but in Italian forum I did not have answers


  • Staff
    Team
    Moderator
    Translator

    @airart mamedb is down and thus, unscrapable. I've seen that sselph just added a mirror for it and enabled SS as source for mame as well. He just hasn't compiled and released a new version with all those changes yet so I can test and upgrade the batch.


  • Staff
    Team
    Moderator
    Translator

    fastscraper has been upgraded and is fully functional again.
    New feature: you can open a folder browser and choose the roms folder of your recalbox directly over the network to scrape from it!
    I have set ScreenScraper as the only source because the quality is superior to any other. You can expand or modify it easily if you wish. Enjoy!



  • Hi,

    How do we download it on github ?

    I don't see any download link there.


  • Staff
    Team
    Moderator
    Translator

    You can select RAW and copy the plain text to a txt file and rename the extension to .bat



  • This tool is fantastic!! Nice work!!

    One problem though, I cannot scrape NeoGeo roms, I get the following output:

    2017/01/10 10:06:56 arcade  srcs: ss
    2017/01/10 10:06:56 console srcs:
    2017/01/10 10:06:56 INFO: Starting: C:\Users[...]\Downloads\neogeo\3countb.zip
    2017/01/10 10:06:56 INFO: Starting: C:\Users[...]\Downloads\neogeo\2020bba.zip
    2017/01/10 10:06:56 INFO: Starting: C:\Users[...]\Downloads\neogeo\2020bbh.zip
    2017/01/10 10:06:56 INFO: Starting: C:\Users[...]\Downloads\neogeo\2020bb.zip
    2017/01/10 10:06:58 ERR: error processing C:\Users[...]\Downloads\neogeo\2020bb.zip: image: unknown format
    2017/01/10 10:06:58 INFO: Starting: C:\Users[...]\Downloads\neogeo\2020bb.zip
    2017/01/10 10:06:58 ERR: error processing C:\Users\\Downloads\neogeo\2020bbh.zip: image: unknown format
    2017/01/10 10:06:58 INFO: Starting: C:\Users[...]\Downloads\neogeo\2020bbh.zip
    2017/01/10 10:06:58 ERR: error processing C:\Users[...]\Downloads\neogeo\3countb.zip: image: unknown format
    2017/01/10 10:06:58 INFO: Starting: C:\Users[...]\Downloads\neogeo\3countb.zip
    2017/01/10 10:06:58 ERR: error processing C:\Users[...]\Downloads\neogeo\2020bba.zip: image: unknown format
    2017/01/10 10:06:58 INFO: Starting: C:\Users[...]\Downloads\neogeo\2020bba.zip
    2017/01/10 10:06:58 ERR: error processing C:\Users[...]\Downloads\neogeo\2020bb.zip: image: unknown format
    2017/01/10 10:06:58 INFO: Starting: C:\Users[...]\Downloads\neogeo\2020bb.zip
    2017/01/10 10:06:58 ERR: error processing C:\Users[...]\Downloads\neogeo\2020bbh.zip: image: unknown format
    2017/01/10 10:06:58 INFO: Starting: C:\Users[...]\Downloads\neogeo\2020bbh.zip
    2017/01/10 10:06:58 ERR: error processing C:\Users[...]\Downloads\neogeo\3countb.zip: image: unknown format
    2017/01/10 10:06:58 INFO: Starting: C:\Users[...]\Downloads\neogeo\3countb.zip
    2017/01/10 10:06:58 ERR: error processing C:\Users[...]\Downloads\neogeo\2020bba.zip: image: unknown format
    2017/01/10 10:06:58 INFO: Starting: C:\Users[...]\Downloads\neogeo\2020bba.zip
    2017/01/10 10:06:58 ERR: error processing C:\Users[...]\Downloads\neogeo\2020bb.zip: image: unknown format
    2017/01/10 10:06:58 INFO: Starting: C:\Users[...]\Downloads\neogeo\alpham2.zip
    panic: runtime error: invalid memory address or nil pointer dereference
    [signal 0xc0000005 code=0x0 addr=0x14 pc=0x408fe7]
    
    goroutine 22 [running]:
    panic(0x7e9580, 0x113a0030)
            /usr/local/go/src/runtime/panic.go:500 +0x331
    main.crawlROMs.func2(0x11472820, 0x114aa900, 0x0, 0x11472810, 0x114755a0, 0x11475520)
            /home/sselph/go/src/github.com/sselph/scraper/scraper.go:295 +0x527
    created by main.crawlROMs
            /home/sselph/go/src/github.com/sselph/scraper/scraper.go:328 +0xbd7
    
    Start    : 10:06:56.62
    Finish   : 10:06:58.67
             ---------------
    Duration : 00:00:02,05
    

  • Staff
    Moderator
    Team

    Hello @maximus37
    the server for arcade is down.



  • @acris Oh, well that explains it :)

    How can I check the status of it? How long has it been down? Is there a site I can keep an eye on?

    Thanks!


  • Staff
    Team
    Moderator
    Translator

    @maximus37 @acris The problem with NEOGEO right now (using SS as source) is explained here: https://github.com/sselph/scraper/issues/128
    So most likely sselph will need to compile a new version with the corrections to allow us to scrape it.



Want to support us ?

246
Online

68895
Users

18997
Topics

141809
Posts

Looks like your connection to Recalbox Forum was lost, please wait while we try to reconnect.