Recalbox Forum

    • Register
    • Login
    • Search
    • Recent
    • Tags
    • recalbox.com
    • Gitlab repository
    • Documentation
    • Discord

    [SOFT] Universal XML Scraper - Windows Scraper for your Recalbox

    Recalbox General
    [soft] universal xml scraper windows
    19
    61
    56722
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • screech
      screech last edited by

      That's it ^^

      thanks @supernature2k πŸ˜‰

      Clean your Romset with Universal ROM Cleaner
      Scrape your Romset with Universal XML Scraper

      1 Reply Last reply Reply Quote 0
      • Tick Lpsk
        Tick Lpsk @supernature2k last edited by

        @supernature2k wow!!!!
        that's work man!
        thank you supernature2k
        also screech for great program

        1 Reply Last reply Reply Quote 0
        • daviduco
          daviduco last edited by daviduco

          Last night i scraped snes succesfully, i'm trying to scrape it now and it fails.

          I can't scrape anything, it scrapes 600 roms in 2 minutes. Has anything changed? I've tried another 9 systems with exact same result.

          [22:11:02] - Recuperation des informations des systemes
          [22:11:03] - Fichier d'informations non valide
          [22:11:03] - Selection du systeme
          [22:11:03] - Recuperation des Profil
          [22:11:03] - Recuperation des champs du profil
          [22:11:03] - Nombre de roms trouvees : 547
          [22:11:03] - Creation des parametres pour le Mix
          [22:11:03] - ---------------------------------------------------------------------------------------
          [22:11:03] - Calcul des Hash en : 0.24s
          [22:11:03] - Recuperation des informations de la Rom no 1 (CRC + SYSTEM)
          [22:11:04] - ()Rom non trouve : 90 Minutes - European Prime Goal (Europe).sfc (Hash : 1AD61BD0)
          [22:11:04] - Fin de l'ecriture des informations de la Rom no 1 en 0.551s
          [22:11:04] - ---------------------------------------------------------------------------------------
          [22:11:04] - Calcul des Hash en : 0.24s
          [22:11:04] - (
          )Rom non trouve : Aaahh!!! Real Monsters (Europe).sfc (Hash : 946313FB)
          [22:11:04] - Fin de l'ecriture des informations de la Rom no 2 en 0.299s
          [22:11:04] - ---------------------------------------------------------------------------------------
          [22:11:04] - Calcul des Hash en : 0.14s
          [22:11:04] - ()Rom non trouve : ACME Animation Factory (Europe).sfc (Hash : AAD3A72D)
          [22:11:04] - Fin de l'ecriture des informations de la Rom no 3 en 0.19s
          [22:11:04] - ---------------------------------------------------------------------------------------
          [22:11:04] - Calcul des Hash en : 0.14s
          [22:11:04] - (
          )Rom non trouve : ActRaiser (Europe).sfc (Hash : 09097B2B)
          [22:11:04] - Fin de l'ecriture des informations de la Rom no 4 en 0.19s
          [22:11:04] - ---------------------------------------------------------------------------------------
          [22:11:05] - Calcul des Hash en : 0.22s
          [22:11:05] - (*)Rom non trouve : Actraiser 2 (Europe) (En,Fr,De).sfc (Hash : 630E16A1)
          [22:11:05] - Fin de l'ecriture des informations de la Rom no 5 en 0.272s
          [22:11:05] - ---------------------------------------------------------------------------------------
          [22:11:05] - Calcul des Hash en : 0.24s
          [22:11:05] - ---------------------------------------------------------------------------------------
          [22:11:05] - Debut de fusion des fichiers
          [22:11:05] - Bilan

          [22:11:05] - Scrape Annule
          [22:11:05] - Roms trouvees :0 / 5
          [22:11:05] - Roms ajoutes :0 / 5
          [22:11:05] - Temps moyen par Rom :0.3s
          [22:11:05] - Temps Total :0h 0m 2s

          1 Reply Last reply Reply Quote 0
          • screech
            screech last edited by

            Sorry. The server provider have problem. It will came back tomorrow.

            Clean your Romset with Universal ROM Cleaner
            Scrape your Romset with Universal XML Scraper

            1 Reply Last reply Reply Quote 0
            • thanosazlin
              thanosazlin last edited by

              just used it , works great. but, i only picked up 9 out of 206 N64 games? my .z64 files all have the name of the game no other special characters or extra text like (USA) or (U) [!] in the name of the roms. just wondering what might be causing them to not get picked up?

              example of one of the roms in the _nintendo 64_missing.txt file

              1080 Snowboarding.z64 08FE81C7 16 777 216 07:33.00 2004-03-28 FA27089C425DBAB99F19245C5C997613

              paradadf 1 Reply Last reply Reply Quote 0
              • paradadf
                paradadf Staff @thanosazlin last edited by

                @thanosazlin please, read the forum:
                https://forum.recalbox.com/topic/3842/how-to-screenscraper

                There you have the explanation why you fon't get roms scraped. Just a hint: the filename must be exact as the one in the database (without extension), when the CRC hash doesn't match.

                This is the thread for the version 2 btw:
                https://forum.recalbox.com/topic/4690/soft-universal-xml-scraper-v2

                If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                1 Reply Last reply Reply Quote 0
                • azvarel
                  azvarel last edited by

                  Suggestion for UXS implementation: when the internet connection fails, instead of redoing the scraper starting again, downloading the images where it left off and then being able to mount the gamelist.xml

                  My connection is 35mbps and the scrapper download is very slow, 40 scrapers for example here gives an estimate of 54 minutes 42 seconds.

                  Zodian Theme | #SNESProject

                  paradadf 1 Reply Last reply Reply Quote 0
                  • paradadf
                    paradadf Staff @azvarel last edited by

                    @azvarel you can change the scrape mode in general configurations from new file to add new game (or something like that). There is no need to scrape everything again.
                    About the speed... try contributing to screenscraper.fr to become more threads πŸ˜‰

                    If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                    azvarel 1 Reply Last reply Reply Quote 0
                    • azvarel
                      azvarel @paradadf last edited by

                      @paradadf But if you drop the internet connection, it does not generate the gamelist.xml file, the program simply gives error and closes. Then this option is not valid.

                      Zodian Theme | #SNESProject

                      1 Reply Last reply Reply Quote 0
                      • screech
                        screech last edited by

                        @azvarel normally there is a timeout function with retry... It try to download 3 times with 20 sec timeout for every file... And if it can't download, it will try the next rom... So it's strange you have a "crash"...

                        40 roms in 54minutes is very longue... (more than a minute by rom :S )

                        normally it's about 1 or 2 sec by roms... (even without multi-thread)

                        Wich rom are you trying to scrape ?
                        Exemple Big BIN or ISO files can take long time but it's not a "scrape" problem, it's an "Hash" problem...
                        Your computeur need to calculate a value based on the file you scrape...
                        If it's a big file and you have a pretty slow computer, it can take lot's of time...

                        For these "type" of file, You need to consider .CUE or .GDI files.
                        UXS will put you BIN in hidden without scrape them if you have a .CUE file with the same name πŸ˜‰

                        Clean your Romset with Universal ROM Cleaner
                        Scrape your Romset with Universal XML Scraper

                        azvarel 2 Replies Last reply Reply Quote 0
                        • azvarel
                          azvarel @screech last edited by

                          @screech my pc no is slow, is AM3+ with 16gb RAM, I'm doing this via wifi connection, it has some influence, I did not put the rpi3 on the router via ethernet cable. This time it was roms of CPS 2 that took time, even today I could not even do those of NEOGEO, simply the program UXS gave error and closed the application, I lost hours trying here and I decided to test the scraper atari 2600 and to my surprise, it worked. I tested initially with 16 roms, and the UXS gave an estimated term of 58 min and 28 sec, time that gives enough to watch an episode of a TV series. 😞

                          Zodian Theme | #SNESProject

                          1 Reply Last reply Reply Quote 0
                          • eb2016
                            eb2016 last edited by

                            Is there a mega noob friendly tutorial for this to help me get started with it?

                            1 Reply Last reply Reply Quote 0
                            • azvarel
                              azvarel @screech last edited by

                              @eb2016 translate the tutorial https://forum.recalbox.com/topic/3489/soft-universal-xml-scraper-portuguΓͺs/

                              @screech I'm running test with ethernet cable to see if it goes faster and still very slow, but so far the UXS did not give error and did not close application

                              Zodian Theme | #SNESProject

                              1 Reply Last reply Reply Quote 0
                              • screech
                                screech last edited by

                                It's reaaly weird that is so slow...
                                Can you check your antivirus and/or firewall ? MAybe it's them slowing down UXS...

                                I just try a scrape on network, and it works fine.. Less than 1sec with a simple screenshot, and between 2 and 4 sec for a classic MIX.

                                Maybe you can try with the "digimirror" function unchecked (in GENERAL CONFIG)

                                Clean your Romset with Universal ROM Cleaner
                                Scrape your Romset with Universal XML Scraper

                                1 Reply Last reply Reply Quote 0
                                • briefer
                                  briefer last edited by

                                  Hi

                                  thank you for the great work. The scrapper works great... except for zxspectrum games. Not even 1 game is found, and I have thousands of rom files.

                                  When I look for the hash numbers in the log, I cannot find any match in the database.

                                  Any help would be greatly appreciated...

                                  1 Reply Last reply Reply Quote 0
                                  • tavareslfelipe
                                    tavareslfelipe last edited by

                                    Is there a way to do this on MacOS? I tried run this scrapper on Wine, but it didn't work.

                                    1 Reply Last reply Reply Quote 0
                                    • DarkMatter
                                      DarkMatter last edited by

                                      First of all thanks for the tutorial.

                                      I'm able to run the scraper, and it finds 100% of the roms πŸ™‚ I went and checked \RECALBOX\share\roms\lynx and I see the downloaded_images folder with the game images, and there is a gameslist.xml file. However when I checked the system the images do not show by any of the games. I tried a restart, and still no luck.

                                      I'm at a loss at how to proceed. I believe when it says "kill emulation station" before scraping that isn't working b/c the frontend is still loaded and showing on the TV screen in the other room while scraping occurs? I'm not familiar with using SecureShell, but I simply connect via network share.

                                      1 Reply Last reply Reply Quote 0
                                      • stamstamstam
                                        stamstamstam last edited by

                                        Hello,
                                        I'm trying to use Universal XML Scraper64, but all I get is a fatal error (problem connecting to the database).
                                        Firewall and Antivirus are clear, no idea what's going on. I'm using WIndows 7 64 pro.
                                        Would someone be kind enough to point me the right direction please? πŸ™‚
                                        Thanks!

                                        1 Reply Last reply Reply Quote 0
                                        • KillerQ
                                          KillerQ last edited by

                                          I seem to be having the EXACT same issue. I have used XML scraper several times in the past with 100% success. Now, for the past few days it will just make a bunch of 0kb png files or no files at all.

                                          Something has definitely changed with the newest update or the server or something.

                                          Even the roms that would scrape a few weeks ago no longer scrape

                                          1 Reply Last reply Reply Quote 0
                                          • screech
                                            screech last edited by

                                            @KillerQ look here : https://forum.recalbox.com/topic/4690/soft-universal-xml-scraper-v2/218

                                            πŸ˜‰

                                            Clean your Romset with Universal ROM Cleaner
                                            Scrape your Romset with Universal XML Scraper

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post

                                            Want to support us ?

                                            68
                                            Online

                                            99.6k
                                            Users

                                            28.1k
                                            Topics

                                            187.1k
                                            Posts

                                            Copyright Β© 2021 recalbox.com