[BATCH] Scrape your roms on your PC - fastscraper
-
what database/website does it get its data from? I'm keen to have a look.
-
@eb2016 all the information is here.
For example:
https://forum.recalbox.com/topic/2594/batch-scrape-your-roms-on-your-pc-fastscraper/9 -
Does this have issues at all with Gameboy Advanced? I just ran it on a bunch of systems and it worked fine but it didn't scrape any of my GBA titles.
-
Please, read the thread if you want to know how everything work. Just a few messages before I wrote this:
https://forum.recalbox.com/topic/2594/batch-scrape-your-roms-on-your-pc-fastscraper/48
It depends on the data source you choose and if the rom hash is in that database. Like all scrapers, it is always better to use no-intro romsets.If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved
-
@paradadf Can you please explain to me what im doing wrong? Ive downloaded scraper.exe and fastscraper.
Ive put them in a folder called "roms" on my desktop and inside said folder are all the systems with roms that i want to scrap. I click on fast scraper, it starts, asks me "which systems do i want to scrape? All? " i enter "all"
then i append the existing game list. hit enter and nothing happens. What am i doing wrong? -
@wggallagher it is impossible that nothing happened... what does the console says? Do you get the duration? Have you downloaded the correct
scraper.exe(64bit version on a 32bit computer)?If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved
-
@paradadf I believe so, I'm running Windows 10. I downloaded the windows386 .zip. should be the right one.
-
@wggallagher what does the console says after you press ENTER on the appending questions?
If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved
-
@wggallagher 64 bit computer
-
@wggallagher nothing. It just goes off screen. Like it closes
-
@wggallagher what happens if you type a specific folder?
If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved
-
@paradadf if I run fast scraper as admin, it says I dont have scraper.exe, even though I do, and asks if if like to go to the website to get it
-
@paradadf if I tell it to scrape nes, instead of all, it does the same thing after the append question. But I can just run scraper.exe and it starts downloading pictures into a "images" folder it creates
-
@wggallagher have you changed anything inside the batch file? Try downloading it again. I have no idea what is happening. Otherwise, it must be something wirh your computers configuration.
If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved
-
@paradadf no, I didn't touch the batch file. I'll try downloading again. I see that it's suppose to produce a images folder and a gameslist.txt. Where do the they go on the memory card, if I get this to work? I know I have to stop emulationstston. I just don't know where to put those 2 new items though when I get them
-
@wggallagher They go exactly as the structure you created on your computer (that should be the same as on Recalbox). Next to the roms goes the
gamelist.xmland the subdirectorydownloaded_images.If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved
-
@paradadf ok, the 2 items created go in the roms folder. OK, thank you very much.
-
@paradadf I redownloaded and same results. Guess im SOL, thanks for your help anyway.
-
Im getting "too many connections" error the first time I use the FastScrapper.
I dont know what to do...... any suggest?
Thanks!!Scraping atari2600 in progress. Please wait... 2016/12/06 03:55:01 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping atari7800 in progress. Please wait... 2016/12/06 03:55:01 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping atarist in progress. Please wait... 2016/12/06 03:55:02 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping cavestory in progress. Please wait... 2016/12/06 03:55:02 INFO: Checking for new hash.csv. 2016/12/06 03:55:02 INFO: hash.csv "e2315a44c3d4c5d200477a109e079f29" up to date. Scraping fba in progress. Please wait... 2016/12/06 03:55:02 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping fba_libretro in progress. Please wait... 2016/12/06 03:55:03 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping fds in progress. Please wait... 2016/12/06 03:55:03 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping gamegear in progress. Please wait... 2016/12/06 03:55:03 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping gb in progress. Please wait... 2016/12/06 03:55:04 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping gba in progress. Please wait... 2016/12/06 03:55:04 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping gbc in progress. Please wait... 2016/12/06 03:55:05 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping gw in progress. Please wait... 2016/12/06 03:55:05 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping lutro in progress. Please wait... 2016/12/06 03:55:05 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping lynx in progress. Please wait... 2016/12/06 03:55:06 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping mame in progress. Please wait... 2016/12/06 03:55:06 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping mastersystem in progress. Please wait... 2016/12/06 03:55:06 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping megadrive in progress. Please wait... 2016/12/06 03:55:07 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping moonlight in progress. Please wait... 2016/12/06 03:55:07 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping msx in progress. Please wait... 2016/12/06 03:55:07 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping msx1 in progress. Please wait... 2016/12/06 03:55:08 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping msx2 in progress. Please wait... 2016/12/06 03:55:08 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping n64 in progress. Please wait... 2016/12/06 03:55:09 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping neogeo in progress. Please wait... 2016/12/06 03:55:09 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping nes in progress. Please wait... 2016/12/06 03:55:09 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping ngp in progress. Please wait... 2016/12/06 03:55:10 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping ngpc in progress. Please wait... 2016/12/06 03:55:10 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping o2em in progress. Please wait... 2016/12/06 03:55:10 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping pcengine in progress. Please wait... 2016/12/06 03:55:11 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping pcenginecd in progress. Please wait... 2016/12/06 03:55:11 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping prboom in progress. Please wait... 2016/12/06 03:55:11 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping psx in progress. Please wait... 2016/12/06 03:55:12 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping scummvm in progress. Please wait... 2016/12/06 03:55:12 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping sega32x in progress. Please wait... 2016/12/06 03:55:13 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping segacd in progress. Please wait... 2016/12/06 03:55:13 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping sg1000 in progress. Please wait... 2016/12/06 03:55:13 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping snes in progress. Please wait... 2016/12/06 03:55:14 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping supergrafx in progress. Please wait... 2016/12/06 03:55:14 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping vectrex in progress. Please wait... 2016/12/06 03:55:14 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping virtualboy in progress. Please wait... 2016/12/06 03:55:15 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping wswan in progress. Please wait... 2016/12/06 03:55:15 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping wswanc in progress. Please wait... 2016/12/06 03:55:15 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping zx81 in progress. Please wait... 2016/12/06 03:55:16 INFO: Checking for new hash.csv. got 429 Too Many Requests response Scraping zxspectrum in progress. Please wait... 2016/12/06 03:55:16 INFO: Checking for new hash.csv. got 429 Too Many Requests response Start : 3:55:00,90 Finish : 3:55:16,88 --------------- Duration : 00:00:15,98 D:\0_COPIA_SEG_RECALBOX\recalbox\roms> -
I'd say you are having too many requests response XD
That depends on the data source. Have you increased the workers? Probably your IP is doing too many requests in a short period of time and the API is blocking you, I don't know.Fastscraper ist just a sort-of interface for sselph's scraper. I don't know exactly what every error displayed by it means, sorry.
If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved
