Scraping Feature Suggested Enhancements
-
Hi all... Just some friendly suggested enhancements regarding the scraping feature as I have been dealing with it for the past few days against a few thousand roms
-
Anyway to make the scraping performance faster? the raspberry pi is hardwired to a 100Mbps connection and just taking forever to download the images and content of each rom
-
Anyway to auto-retry network errors on its own? I find that in a given 12 hour scraping process that the connection times out a dozen or so times for whatever reason and requires human intervention to retry. I don't want to babysit it, so I would rather have a counter that I can set to re-try on its own so that it can continue the process
-
Anyway for the software to send an sms text or network notification when the scraping process has been stopped due to re-try prompt or that the process has been completed so that I don't have to baby sit it.
-
A way for the software to mark the roms that it has already checked and could not find any images for them. That way if a scraping process is re-initiated, that it will skip those instead of re-attempting them again.
-
-
@tdelios I am not sure if this will help you, but I noticed that scraping using the built-in scraper while the RPi is set to overclock will result in freezing.
I used to freeze up after 100 or so games while I was overclocked, but shutting it off I was able to scrape my entire library.
Personally, I found more success, faster and more accurate scraping using a PC based scraper: https://github.com/recalbox/recalbox-os/wiki/Scrape-your-games-with-your-PC-(EN)
-
-
@screech how do I install this on my Windows 10 machine? I downloaded it and unzipped it to a folder and attempted to run the 'scraper' or 'scraper64' executable file, but nothing happens?
-
Hum.. I think you downloaded the wrong file
normally you have nothing to unzip
try this link : https://github.com/Universal-Rom-Tools/Universal-XML-Scraper/releases/download/2.0.0.5/Universal_XML_Scraper64.exe
(for information scraper.exe and scraper64.exe are the "scrape engine" so they do nothing without the main exe
)
