Navigation

    Recalbox Forum

    • Register
    • Login
    • Search
    • Recent
    • Tags
    • recalbox.com
    • Gitlab repository
    • Documentation
    • Discord

    Scraping Feature Suggested Enhancements

    Recalbox General
    scraping feature suggested enhancements
    3
    5
    1769
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • tdelios
      tdelios last edited by

      Hi all... Just some friendly suggested enhancements regarding the scraping feature as I have been dealing with it for the past few days against a few thousand roms

      1. Anyway to make the scraping performance faster? the raspberry pi is hardwired to a 100Mbps connection and just taking forever to download the images and content of each rom

      2. Anyway to auto-retry network errors on its own? I find that in a given 12 hour scraping process that the connection times out a dozen or so times for whatever reason and requires human intervention to retry. I don't want to babysit it, so I would rather have a counter that I can set to re-try on its own so that it can continue the process

      3. Anyway for the software to send an sms text or network notification when the scraping process has been stopped due to re-try prompt or that the process has been completed so that I don't have to baby sit it.

      4. A way for the software to mark the roms that it has already checked and could not find any images for them. That way if a scraping process is re-initiated, that it will skip those instead of re-attempting them again.

      sag508 1 Reply Last reply Reply Quote 1
      • sag508
        sag508 @tdelios last edited by

        @tdelios I am not sure if this will help you, but I noticed that scraping using the built-in scraper while the RPi is set to overclock will result in freezing.

        I used to freeze up after 100 or so games while I was overclocked, but shutting it off I was able to scrape my entire library.

        Personally, I found more success, faster and more accurate scraping using a PC based scraper: https://github.com/recalbox/recalbox-os/wiki/Scrape-your-games-with-your-PC-(EN)

        1 Reply Last reply Reply Quote 0
        • screech
          screech last edited by

          or try https://forum.recalbox.com/topic/4690/soft-universal-xml-scraper-v2/

          Clean your Romset with Universal ROM Cleaner
          Scrape your Romset with Universal XML Scraper

          tdelios 1 Reply Last reply Reply Quote 0
          • tdelios
            tdelios @screech last edited by

            @screech how do I install this on my Windows 10 machine? I downloaded it and unzipped it to a folder and attempted to run the 'scraper' or 'scraper64' executable file, but nothing happens?

            1 Reply Last reply Reply Quote 0
            • screech
              screech last edited by

              Hum.. I think you downloaded the wrong file 😉

              normally you have nothing to unzip 😉

              try this link : https://github.com/Universal-Rom-Tools/Universal-XML-Scraper/releases/download/2.0.0.5/Universal_XML_Scraper64.exe

              (for information scraper.exe and scraper64.exe are the "scrape engine" so they do nothing without the main exe 😉 )

              Clean your Romset with Universal ROM Cleaner
              Scrape your Romset with Universal XML Scraper

              1 Reply Last reply Reply Quote 0
              • First post
                Last post

              Want to support us ?

              83
              Online

              87.8k
              Users

              25.8k
              Topics

              178.7k
              Posts

              Copyright © 2021 recalbox.com