Recalbox Forum

    • Register
    • Login
    • Search
    • Recent
    • Tags
    • recalbox.com
    • Gitlab repository
    • Documentation
    • Discord

    [SOFT] Universal XML Scraper - Scrappez vos roms depuis Windows (Scrape your roms from Windows)

    Community projects
    [soft] universal xml scraper scrappez
    73
    656
    482343
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • screech
      screech last edited by

      It's OK !! the 1.4.0.2 will be ok without the "PAUSE" message :S sorry for the inconvient...

      Clean your Romset with Universal ROM Cleaner
      Scrape your Romset with Universal XML Scraper

      Nachtgarm 1 Reply Last reply Reply Quote 0
      • Nachtgarm
        Nachtgarm @screech last edited by Nachtgarm

        Hey @screech. Please add a "Start EmulationStation" command as well. Rebooting the whole thing after every scrape is ok, but not necessary, right 😉

        Further it would be great to NOT Scrape the savegames. E.g: Advance Wars.zip and Advance Wars.sav will both be scraped as Games and therefore displayed in ES. That's actually a bug, not a feature 😉

        MSX is recognized as MAME with the Autoconfig 😞

        1 Reply Last reply Reply Quote 0
        • supernature2k
          supernature2k last edited by

          @Nachtgarm , there's already a way to filter these out 🙂
          Either in confug file or in advanced settings menu.

          Pi powered NES | Gameboy HD | RecalStation | RecalDrive
          Upvote messages if it has been useful ;)

          Nachtgarm 1 Reply Last reply Reply Quote 0
          • Nachtgarm
            Nachtgarm @supernature2k last edited by

            @supernature2k I acutally haven't seen that until now 🙂 Thanks for the heads up, but still MSX is recognized as Mame and a "Start ES" would be great too 😉

            1 Reply Last reply Reply Quote 0
            • screech
              screech last edited by

              I'm on it 😉

              Just a question : What is the command line to start emulationstation ?

              Clean your Romset with Universal ROM Cleaner
              Scrape your Romset with Universal XML Scraper

              1 Reply Last reply Reply Quote 0
              • acris
                acris last edited by

                /etc/init.d/S31emulationstation start
                /etc/init.d/S31emulationstation stop
                /etc/init.d/S31emulationstation restart

                1 Reply Last reply Reply Quote 0
                • screech
                  screech last edited by

                  Thanks @acris ^^

                  Clean your Romset with Universal ROM Cleaner
                  Scrape your Romset with Universal XML Scraper

                  1 Reply Last reply Reply Quote 0
                  • screech
                    screech last edited by screech

                    @naitreia I debbugged the Time Out... Now if you have a time Out it try to do the next rom and so on...

                    @sassa J'ai corrigé le fichier missing. Je ne sais pas pourquoi mais il y avait boulette sur sa génération, maintenant ça marche 😉

                    @Nachtgarm MSX is now MSX 😉 and Start ES is added in the SSH menu 😉

                    For every one, the Multilinguale integration in Screenscraper is huge now 😉 so there are the info in Universal XML Scraper V1.4.0.4 😉

                    (Some modification : Name of the game, description, genre, date, etc... Are now in French, English, German, Portuguese, and Spanish... When it's in Screenscraper 😉 So don't hesitate to contribute in you own language... Translation of the interface of Screnscraper is in progress too ^^)

                    Clean your Romset with Universal ROM Cleaner
                    Scrape your Romset with Universal XML Scraper

                    1 Reply Last reply Reply Quote 1
                    • naitreia
                      naitreia last edited by

                      Hi Screech,

                      here are the last lines from the error log after he says Database would be down:

                      [17:25:30] - ---------------------------------------------------------------------------------------
                      [17:25:30] - Calcul des Hash en : 0.02s
                      [17:25:30] - Recuperation des informations de la Rom no 2087 (CRC + SYSTEM)
                      [17:25:31] - --Recuperation des informations de la Rom no 2087 (CRC)
                      [17:25:31] - Recuperation des informations de la Rom no 2087 (FileName + SYSTEM)
                      [17:34:38] - ---------------------------------------------------------------------------------------
                      [17:34:38] - Debut de fusion des fichiers
                      [17:37:13] - Fin de fusion des fichiers en 155.31s
                      [17:37:13] - Bilan

                      [17:37:14] - Scrape Annule
                      [17:37:14] - Roms trouvees :2086 / 2086
                      [17:37:14] - Roms ajoutes :2086 / 2086
                      [17:37:14] - Temps moyen par Rom :6.801s
                      [17:37:14] - Temps Total :4h 8m 22s
                      [17:37:27] - Fermeture du Soft

                      Hope it helps you and you can fix it.

                      1 Reply Last reply Reply Quote 0
                      • screech
                        screech last edited by

                        Strange.. I see nothing anormal...
                        17:25:31 -> getting the last rom information
                        17:34:38 -> Fusionning every xml in one gamelist.xml
                        17:37:13 -> End of fusionning + calculate the "bilan"

                        The only thing strange is : [17:37:14] - Scrape Annule
                        who mean you canceled the scrape.

                        But no Timeout... (if it was one, you had a TimeOut Line in the log)

                        You had an error message ?

                        Clean your Romset with Universal ROM Cleaner
                        Scrape your Romset with Universal XML Scraper

                        1 Reply Last reply Reply Quote 0
                        • naitreia
                          naitreia last edited by

                          Hi,

                          you can reproduce it by scraping round about 4-5 hours.
                          I think after 2k roms or so it will hang up.

                          What happens is, that the scrape ends and you get the error-message "Database are down" or something like this.

                          I tried it 3 times always the same result.

                          1 Reply Last reply Reply Quote 0
                          • naitreia
                            naitreia last edited by

                            Oh and im not canceled the scrape!
                            Message database are down coming and thats it.

                            1 Reply Last reply Reply Quote 0
                            • Duss
                              Duss last edited by

                              Bonjour, j'ai un problème pour scraper la lynx et je ne trouve pas la solution.
                              Ca trouve toutes les roms mais ca ne scrape pas comme ci il n'y avais aucune base de donné pour la lynx alors que c'est quasi complet sur screenscraper ? Mes roms sont pourtant bien nommées et je n'ai ce problème que pour la lynx ...
                              Si une âme charitable pourrait m'aider, merci d'avance.

                              1 Reply Last reply Reply Quote 0
                              • screech
                                screech last edited by

                                @Duss Peux tu poster une partie de ton fichier log ? (10 lignes du début et 10 lignes de la fin 😉 )

                                Histoire de voir si la recherche se passe bien au bon endroit ?
                                (Normalement il fait 3 types de recherche :
                                (CRC + SYSTEM)
                                (CRC)
                                (FileName + SYSTEM)

                                Il devrait au moins trouver ^^ sinon ça veut dire :
                                Que tes roms n'ont pas de CRC ou de nom de fichier (sans extension) connu par Screenscraper

                                Clean your Romset with Universal ROM Cleaner
                                Scrape your Romset with Universal XML Scraper

                                1 Reply Last reply Reply Quote 0
                                • Duss
                                  Duss last edited by

                                  [00:49:06] - Version : 1.4.0.4
                                  [00:49:06] - Langue Selectionnee : 040c
                                  [00:49:06] - Recuperation des Profil
                                  [00:49:13] - SSH : Stop Emulationstation OK
                                  [00:57:47] - Recuperation des informations des systemes
                                  [00:57:54] - Selection du systeme
                                  [00:57:55] - Systeme selectionne : Super Nintendo
                                  [00:57:55] - Recuperation des Profil
                                  [00:57:55] - Recuperation des champs du profil
                                  [00:57:56] - Nombre de roms trouvees : 2199
                                  [00:57:56] - Creation des parametres pour le Mix

                                  ...........................

                                  [01:14:06] - Calcul des Hash en : 0.07s
                                  [01:14:06] - Recuperation des informations de la Rom no 68 (CRC + SYSTEM)
                                  [01:14:06] - --Recuperation des informations de la Rom no 68 (CRC)
                                  [01:14:06] - Recuperation des informations de la Rom no 68 (FileName + SYSTEM)
                                  [01:14:06] - (*)Rom non trouve : Zarlor Mercenary (USA, Europe).zip (Hash : 07087967)
                                  [01:14:06] - Fin de l'ecriture des informations de la Rom no 68 en 0.889s
                                  [01:14:06] - ---------------------------------------------------------------------------------------
                                  [01:14:06] - Debut de fusion des fichiers
                                  [01:14:06] - Bilan

                                  [01:14:08] - Scrape Termine
                                  [01:14:08] - Roms trouvees :0 / 68
                                  [01:14:08] - Roms ajoutes :0 / 68
                                  [01:14:08] - Temps moyen par Rom :0.915s
                                  [01:14:08] - Temps Total :0h 1m 9s

                                  1 Reply Last reply Reply Quote 0
                                  • Duss
                                    Duss last edited by

                                    Je t'avoue que je comprends pas grand chose mais je viens de voir que le fichier gamelist.xml de la Lynx est vide même après le crape.

                                    1 Reply Last reply Reply Quote 0
                                    • Duss
                                      Duss last edited by

                                      Et ca c'est peut etre mieux ^^'

                                      [01:12:57] - Recuperation des informations des systemes
                                      [01:13:03] - Selection du systeme
                                      [01:13:04] - Systeme selectionne : Camputers Lynx
                                      [01:13:04] - Recuperation des Profil
                                      [01:13:04] - Recuperation des champs du profil
                                      [01:13:04] - Nombre de roms trouvees : 68
                                      [01:13:04] - Creation des parametres pour le Mix
                                      [01:13:04] - ---------------------------------------------------------------------------------------
                                      [01:13:04] - Calcul des Hash en : 0.1s
                                      [01:13:04] - Recuperation des informations de la Rom no 1 (CRC + SYSTEM)
                                      [01:13:04] - --Recuperation des informations de la Rom no 1 (CRC)
                                      [01:13:05] - Recuperation des informations de la Rom no 1 (FileName + SYSTEM)
                                      [01:13:05] - (*)Rom non trouve : A.P.B. - All Points Bulletin (USA, Europe).zip (Hash : 194ABA16)
                                      [01:13:05] - Fin de l'ecriture des informations de la Rom no 1 en 0.892s

                                      1 Reply Last reply Reply Quote 0
                                      • screech
                                        screech last edited by

                                        Alors, en fait tu as fait un scrape super Nes avant ^^ du coup le début correspond au scrape super Nes ^^

                                        Par contre, la fin, j'ai ta dernière rom lynx, qui est bien référencé au niveau du nom (mais pas au niveau du Hash) sur ScreenScraper.

                                        Peux tu quitter le logiciel, le relancer, allez dans "configuration" / bouton "Avancée" et mettre "RechSYS" à 0

                                        Du coup ça te demandera ton système avant le scrape, et il faut bien choisir Lynx .

                                        Il devrait au moins te trouver la dernière 😉

                                        Clean your Romset with Universal ROM Cleaner
                                        Scrape your Romset with Universal XML Scraper

                                        1 Reply Last reply Reply Quote 0
                                        • screech
                                          screech last edited by

                                          Ah oui !! c'est mieux ^^

                                          [01:13:04] - Systeme selectionne : Camputers Lynx

                                          Tu ne choisi pas le bon système (ou alors c'est mon automatisation ^^)

                                          Il faut prendre "Lynx" pas "Camputers Lynx"

                                          Clean your Romset with Universal ROM Cleaner
                                          Scrape your Romset with Universal XML Scraper

                                          1 Reply Last reply Reply Quote 0
                                          • Duss
                                            Duss last edited by

                                            J'ai fait mais je ne trouve qu'un seul Lynx malheureusement ^^''
                                            et toujours le même résultat ...

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post

                                            Want to support us ?

                                            90
                                            Online

                                            99.6k
                                            Users

                                            28.1k
                                            Topics

                                            187.1k
                                            Posts

                                            Copyright © 2021 recalbox.com