Recalbox Forum

    • Register
    • Login
    • Search
    • Recent
    • Tags
    • recalbox.com
    • Gitlab repository
    • Documentation
    • Discord

    [BATCH] Scrape your roms on your PC - fastscraper

    Community projects
    multiconsole scraper fastscraper
    37
    221
    240634
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • airart
      airart last edited by

      alt text

      not working xml syntax error on line 1

      paradadf 1 Reply Last reply Reply Quote 0
      • paradadf
        paradadf Staff @airart last edited by paradadf

        @airart what is not working? where do you get that error? what version of sselph's scraper do you have?

        If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

        airart 1 Reply Last reply Reply Quote 0
        • airart
          airart @paradadf last edited by

          @paradadf sselph v1.2.0
          i'm in pc platform not connected to recalbox because at home i have a mac system
          alt text

          paradadf 1 Reply Last reply Reply Quote 0
          • paradadf
            paradadf Staff @airart last edited by paradadf

            @airart are you running the batch as administrator? If so, don't do it.
            What was that about a pc and a mac? I didn't understand.

            FYI there is a new version for macOS:
            https://forum.recalbox.com/topic/5745/bash-scrape-your-roms-on-macos-and-linux-fastscraper

            If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

            airart 1 Reply Last reply Reply Quote 0
            • airart
              airart @paradadf last edited by

              @paradadf said in [BATCH] Scrape your roms on your PC - fastscraper:

              @airart are you running the batch as administrator? If so, don't do it.
              What was that about a pc and a mac? I didn't understand.

              FYI there is a new version for macOS:
              https://forum.recalbox.com/topic/5745/bash-scrape-your-roms-on-macos-and-linux-fastscraper

              I 'm non running as administrator
              I said that on mac not think there was the possibility

              paradadf 1 Reply Last reply Reply Quote 0
              • paradadf
                paradadf Staff @airart last edited by

                @airart are you trying to append an existing gamelist?

                If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                airart 1 Reply Last reply Reply Quote 0
                • airart
                  airart @paradadf last edited by

                  @paradadf said in [BATCH] Scrape your roms on your PC - fastscraper:

                  @airart are you trying to append an existing gamelist?

                  I asked him to create a new one

                  airart 1 Reply Last reply Reply Quote 0
                  • airart
                    airart @airart last edited by

                    @airart said in [BATCH] Scrape your roms on your PC - fastscraper:

                    @paradadf said in [BATCH] Scrape your roms on your PC - fastscraper:

                    @airart are you trying to append an existing gamelist?

                    I asked him to create a new one

                    The problem may be that they are not connected to recalbox?

                    paradadf 1 Reply Last reply Reply Quote 0
                    • paradadf
                      paradadf Staff @airart last edited by

                      @airart no, I never scrape over the network, but on my pc. I have no idea what is going on, but I would try to erase the gamelist.xml and download_images folder and scrape again. I still believe you are executing the batch with administrator privilege and thus the error. Look at the title bar... it says "Amministratore".

                      If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                      airart 1 Reply Last reply Reply Quote 0
                      • airart
                        airart @paradadf last edited by

                        @paradadf said in [BATCH] Scrape your roms on your PC - fastscraper:

                        I have no idea what is going on, but I would try to erase the gamelist.xml and download_images folder and scrape again. I still believe you are executing the batch with administrator privilege and thus the error. Look at the title bar... it says "Amministratore".

                        I know , but I do not understand how to do, I did not set the administrative mode

                        1 Reply Last reply Reply Quote 0
                        • rsn8887
                          rsn8887 last edited by paradadf

                          Worked two days ago, now broken:
                          ERR: error processing H:\EmuRoms\Recalbox\recalbox\roms\gba\2 Game Pack! - Matchbox Missions - Emergency Response + Air, Land and Sea Rescue (USA).zip: 504 from $s%!(EXTRA string=http://mirror.screenscraper.fr/api/mediaJeu.php?crc=&devid=&devpassword=&jeuid=83014&maxwidth=400&md5=&media=ss&mediaformat=png&sha1=&softname=sselph%2Fscraper&systemeid=12)
                          panic: runtime error: invalid memory address or nil pointer dereference
                          [signal 0xc0000005 code=0x0 addr=0x14 pc=0x408fe7]

                          goroutine 31 [running]:
                          panic(0x7e9580, 0x1137e030)
                          /usr/local/go/src/runtime/panic.go:500 +0x331
                          main.crawlROMs.func2(0x120b29d0, 0x11dcb540, 0x0, 0x120b29c0, 0x1287e0c0, 0x1287e040)
                          /home/sselph/go/src/github.com/sselph/scraper/scraper.go:295 +0x527
                          created by main.crawlROMs
                          /home/sselph/go/src/github.com/sselph/scraper/scraper.go:328 +0xbd7

                          paradadf 1 Reply Last reply Reply Quote 0
                          • paradadf
                            paradadf Staff @rsn8887 last edited by paradadf

                            @rsn8887 I read on the UXS thread that gba was having problems the last couple of days... Do you still have this problem? Does it happen with any system or only gba?
                            I've also encounter some single roms which had wrong data on SS and thus "broke" the scraping process.
                            So please,

                            • try to scrape gba again
                            • try scraping any other system to see if you still get an error
                            • if other systems work and if the problematic rom is one of the first ones, try removing it. (And give me the name)

                            I'n not on my computer right now, so I can't test.

                            Update: This is what I had read

                            https://github.com/Universal-Rom-Tools/Universal-XML-Scraper/issues/142

                            If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                            1 Reply Last reply Reply Quote 0
                            • rsn8887
                              rsn8887 last edited by

                              @paradadf
                              Other systems show the same problem, but some don't. For example gb scraped fine completely through all roms, but gba and n64 didn't. I tried to remove a few roms where the error appears, but then it just happens at a later rom. Judging from that 504 error message, it seems to have something to do with mirror.screenscraper.fr . As I said it all worked perfectly on the weekend a few days ago.

                              paradadf 1 Reply Last reply Reply Quote 0
                              • paradadf
                                paradadf Staff @rsn8887 last edited by paradadf

                                @rsn8887 sselph's scraper uses no mirror. It has something to do with the database and not with the scraper itself. There is nothing I can do. You can complain about it on IRC XD

                                Update: I just tried it and saw the mirror thing you mentioned. But no idea what that is! I'll talk with the people there and ask.

                                Update 2: some kind of dns problem. They already are aware of it and are working on it. So just be patient 😉

                                If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                                1 Reply Last reply Reply Quote 0
                                • screech
                                  screech last edited by

                                  Normally it's OK now 😉

                                  Clean your Romset with Universal ROM Cleaner
                                  Scrape your Romset with Universal XML Scraper

                                  paradadf 1 Reply Last reply Reply Quote 0
                                  • paradadf
                                    paradadf Staff @screech last edited by

                                    @screech thanks for the heads up!

                                    If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                                    1 Reply Last reply Reply Quote 0
                                    • rsn8887
                                      rsn8887 last edited by

                                      Yup it works just fine again 🙂

                                      1 Reply Last reply Reply Quote 0
                                      • rsn8887
                                        rsn8887 last edited by

                                        Well, I was a bit fast now there are problems with N64 scraping. It gives "unknown image format" errors. Worked fine 2 hours ago.

                                        "GDB" scraping works fine, so it is again some problem with "ss" (maybe ss ist just inherently unstable?)

                                        1 Reply Last reply Reply Quote 0
                                        • paradadf
                                          paradadf Staff last edited by

                                          I just think they are having dns problems lately. Personally, I'd rather wait until it is working again (usually just a matter of hours) instead of using other sources. No other database can compete with SS in that sense. And gdb is definitely not more stable 🙂

                                          If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                                          1 Reply Last reply Reply Quote 0
                                          • faxesystem
                                            faxesystem last edited by

                                            I have the following problem. When setting 3dboxart as prefered cover art it get's downloaded when scraping for a single system. When I choose all 3dboxart gets scraped for gba but not for any other system. For all other console systems the 2dboxart gets downloaded. Can anyone confirm this?

                                            Greetings,

                                            Faxe

                                            paradadf 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post

                                            Want to support us ?

                                            81
                                            Online

                                            99.6k
                                            Users

                                            28.1k
                                            Topics

                                            187.1k
                                            Posts

                                            Copyright © 2021 recalbox.com