Scraping Feature Suggested Enhancements

[tdelios]

Hi all... Just some friendly suggested enhancements regarding the scraping feature as I have been dealing with it for the past few days against a few thousand roms

1. Anyway to make the scraping performance faster? the raspberry pi is hardwired to a 100Mbps connection and just taking forever to download the images and content of each rom

2. Anyway to auto-retry network errors on its own? I find that in a given 12 hour scraping process that the connection times out a dozen or so times for whatever reason and requires human intervention to retry. I don't want to babysit it, so I would rather have a counter that I can set to re-try on its own so that it can continue the process

3. Anyway for the software to send an sms text or network notification when the scraping process has been stopped due to re-try prompt or that the process has been completed so that I don't have to baby sit it.

4. A way for the software to mark the roms that it has already checked and could not find any images for them. That way if a scraping process is re-initiated, that it will skip those instead of re-attempting them again.

[sag508]

@tdelios I am not sure if this will help you, but I noticed that scraping using the built-in scraper while the RPi is set to overclock will result in freezing.

I used to freeze up after 100 or so games while I was overclocked, but shutting it off I was able to scrape my entire library.

Personally, I found more success, faster and more accurate scraping using a PC based scraper: https://github.com/recalbox/recalbox-os/wiki/Scrape-your-games-with-your-PC-(EN)

[screech]

or try https://forum.recalbox.com/topic/4690/soft-universal-xml-scraper-v2/

[tdelios]

@screech how do I install this on my Windows 10 machine? I downloaded it and unzipped it to a folder and attempted to run the 'scraper' or 'scraper64' executable file, but nothing happens?

[screech]

Hum.. I think you downloaded the wrong file

normally you have nothing to unzip

try this link : https://github.com/Universal-Rom-Tools/Universal-XML-Scraper/releases/download/2.0.0.5/Universal_XML_Scraper64.exe

(for information scraper.exe and scraper64.exe are the "scrape engine" so they do nothing without the main exe )