Recalbox Forum

    • Register
    • Login
    • Search
    • Recent
    • Tags
    • recalbox.com
    • Gitlab repository
    • Documentation
    • Discord

    [SOFT] Universal XML Scraper - Scrappez vos roms depuis Windows (Scrape your roms from Windows)

    Community projects
    [soft] universal xml scraper scrappez
    73
    656
    482432
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • supernature2k
      supernature2k last edited by

      les pictos sont récupérables par l'API?

      Si on pouvait mettre ça dans ES... le top!

      Pi powered NES | Gameboy HD | RecalStation | RecalDrive
      Upvote messages if it has been useful ;)

      1 Reply Last reply Reply Quote 0
      • screech
        screech last edited by

        Question transfered to MarbleMad ^^
        And the correction too 😛

        Clean your Romset with Universal ROM Cleaner
        Scrape your Romset with Universal XML Scraper

        1 Reply Last reply Reply Quote 0
        • screech
          screech last edited by

          WARNING

          There is a big modification on Screenscraper.
          Now you can have data in french, english, but also german, spanish, and portuguese !!!

          So there is a modification of Universal XML Scraper to take that in charge 😉

          Please Use the very last version
          (If not only 'genre' doesn't work ^^ But I don't know if there are no side effect 😛 )

          Clean your Romset with Universal ROM Cleaner
          Scrape your Romset with Universal XML Scraper

          1 Reply Last reply Reply Quote 0
          • sassa
            sassa last edited by sassa

            merci pour l'info pour les jeux psx effectivement il m'en a trouvé mais très peu

            par contre, quand le logiciel scrapp un dossier, il me crée pas le fichier missing pour me dire quels roms n'ont pas été trouvé

            Y'a une option à cocher pour l'avoir ? Avec la version 1.3.0.9 je l'avais (oui j'étais pas à jour lol) mais là j'utilise la 1.3.0.15 et il ne se créer plus

            edit : c'est bon j'ai trouvé il fallait cocher non trouvé dans configuration pffffff

            va falloir que je rescanne tout grrrrr

            1 Reply Last reply Reply Quote 0
            • supernature2k
              supernature2k last edited by

              Non non, non trouvé sert à ajouter les roms nin trouvées sur screenscraper dans ton gamelist.

              Pi powered NES | Gameboy HD | RecalStation | RecalDrive
              Upvote messages if it has been useful ;)

              1 Reply Last reply Reply Quote 0
              • screech
                screech last edited by

                Ahh, mais il y a peut etre un bug du coup ^^ (c'est possible qu'il ne créé le fichier missing que quand tu coche non trouvé)

                Clean your Romset with Universal ROM Cleaner
                Scrape your Romset with Universal XML Scraper

                1 Reply Last reply Reply Quote 0
                • sassa
                  sassa last edited by

                  la plupart ce sont des roms existantes mais qui ont été patché en langue fr donc le crc ne correspond plus

                  mais je sais que pour pas mal de dossier il en manquait et justement je sais pas lesquels c'est du coup car j'ai lancé tout à la suite sans vérifier

                  oui j'avais bien compris que ça servait à ajouter sur le site 😉

                  Je débute mais en tout cas ça aide ce genre de logiciel merci pour le boulot en tout cas

                  oui peut être un bug car ayant encore la version 9 de côté j'ai vérifié la case n'était pas coché je vais re essayer comme j'ai pas fini de scrapper

                  1 Reply Last reply Reply Quote 0
                  • naitreia
                    naitreia last edited by

                    The new Scraper dont work correct.
                    After every Image it says "Pause"!

                    1 Reply Last reply Reply Quote 0
                    • screech
                      screech last edited by

                      Arg !!! Yes 😞 don't use the last version right now 😞 I forgot a debug message 😢

                      I correct it tomorrow at the first hour I promise (can't do it right now... really sorry 😢 )

                      Clean your Romset with Universal ROM Cleaner
                      Scrape your Romset with Universal XML Scraper

                      1 Reply Last reply Reply Quote 0
                      • screech
                        screech last edited by

                        It's OK !! the 1.4.0.2 will be ok without the "PAUSE" message :S sorry for the inconvient...

                        Clean your Romset with Universal ROM Cleaner
                        Scrape your Romset with Universal XML Scraper

                        Nachtgarm 1 Reply Last reply Reply Quote 0
                        • Nachtgarm
                          Nachtgarm @screech last edited by Nachtgarm

                          Hey @screech. Please add a "Start EmulationStation" command as well. Rebooting the whole thing after every scrape is ok, but not necessary, right 😉

                          Further it would be great to NOT Scrape the savegames. E.g: Advance Wars.zip and Advance Wars.sav will both be scraped as Games and therefore displayed in ES. That's actually a bug, not a feature 😉

                          MSX is recognized as MAME with the Autoconfig 😞

                          1 Reply Last reply Reply Quote 0
                          • supernature2k
                            supernature2k last edited by

                            @Nachtgarm , there's already a way to filter these out 🙂
                            Either in confug file or in advanced settings menu.

                            Pi powered NES | Gameboy HD | RecalStation | RecalDrive
                            Upvote messages if it has been useful ;)

                            Nachtgarm 1 Reply Last reply Reply Quote 0
                            • Nachtgarm
                              Nachtgarm @supernature2k last edited by

                              @supernature2k I acutally haven't seen that until now 🙂 Thanks for the heads up, but still MSX is recognized as Mame and a "Start ES" would be great too 😉

                              1 Reply Last reply Reply Quote 0
                              • screech
                                screech last edited by

                                I'm on it 😉

                                Just a question : What is the command line to start emulationstation ?

                                Clean your Romset with Universal ROM Cleaner
                                Scrape your Romset with Universal XML Scraper

                                1 Reply Last reply Reply Quote 0
                                • acris
                                  acris last edited by

                                  /etc/init.d/S31emulationstation start
                                  /etc/init.d/S31emulationstation stop
                                  /etc/init.d/S31emulationstation restart

                                  1 Reply Last reply Reply Quote 0
                                  • screech
                                    screech last edited by

                                    Thanks @acris ^^

                                    Clean your Romset with Universal ROM Cleaner
                                    Scrape your Romset with Universal XML Scraper

                                    1 Reply Last reply Reply Quote 0
                                    • screech
                                      screech last edited by screech

                                      @naitreia I debbugged the Time Out... Now if you have a time Out it try to do the next rom and so on...

                                      @sassa J'ai corrigé le fichier missing. Je ne sais pas pourquoi mais il y avait boulette sur sa génération, maintenant ça marche 😉

                                      @Nachtgarm MSX is now MSX 😉 and Start ES is added in the SSH menu 😉

                                      For every one, the Multilinguale integration in Screenscraper is huge now 😉 so there are the info in Universal XML Scraper V1.4.0.4 😉

                                      (Some modification : Name of the game, description, genre, date, etc... Are now in French, English, German, Portuguese, and Spanish... When it's in Screenscraper 😉 So don't hesitate to contribute in you own language... Translation of the interface of Screnscraper is in progress too ^^)

                                      Clean your Romset with Universal ROM Cleaner
                                      Scrape your Romset with Universal XML Scraper

                                      1 Reply Last reply Reply Quote 1
                                      • naitreia
                                        naitreia last edited by

                                        Hi Screech,

                                        here are the last lines from the error log after he says Database would be down:

                                        [17:25:30] - ---------------------------------------------------------------------------------------
                                        [17:25:30] - Calcul des Hash en : 0.02s
                                        [17:25:30] - Recuperation des informations de la Rom no 2087 (CRC + SYSTEM)
                                        [17:25:31] - --Recuperation des informations de la Rom no 2087 (CRC)
                                        [17:25:31] - Recuperation des informations de la Rom no 2087 (FileName + SYSTEM)
                                        [17:34:38] - ---------------------------------------------------------------------------------------
                                        [17:34:38] - Debut de fusion des fichiers
                                        [17:37:13] - Fin de fusion des fichiers en 155.31s
                                        [17:37:13] - Bilan

                                        [17:37:14] - Scrape Annule
                                        [17:37:14] - Roms trouvees :2086 / 2086
                                        [17:37:14] - Roms ajoutes :2086 / 2086
                                        [17:37:14] - Temps moyen par Rom :6.801s
                                        [17:37:14] - Temps Total :4h 8m 22s
                                        [17:37:27] - Fermeture du Soft

                                        Hope it helps you and you can fix it.

                                        1 Reply Last reply Reply Quote 0
                                        • screech
                                          screech last edited by

                                          Strange.. I see nothing anormal...
                                          17:25:31 -> getting the last rom information
                                          17:34:38 -> Fusionning every xml in one gamelist.xml
                                          17:37:13 -> End of fusionning + calculate the "bilan"

                                          The only thing strange is : [17:37:14] - Scrape Annule
                                          who mean you canceled the scrape.

                                          But no Timeout... (if it was one, you had a TimeOut Line in the log)

                                          You had an error message ?

                                          Clean your Romset with Universal ROM Cleaner
                                          Scrape your Romset with Universal XML Scraper

                                          1 Reply Last reply Reply Quote 0
                                          • naitreia
                                            naitreia last edited by

                                            Hi,

                                            you can reproduce it by scraping round about 4-5 hours.
                                            I think after 2k roms or so it will hang up.

                                            What happens is, that the scrape ends and you get the error-message "Database are down" or something like this.

                                            I tried it 3 times always the same result.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post

                                            Want to support us ?

                                            94
                                            Online

                                            99.6k
                                            Users

                                            28.1k
                                            Topics

                                            187.1k
                                            Posts

                                            Copyright © 2021 recalbox.com