Recalbox Forum

    • Register
    • Login
    • Search
    • Recent
    • Tags
    • recalbox.com
    • Gitlab repository
    • Documentation
    • Discord

    [BATCH] Scrape your roms on your PC - fastscraper

    Community projects
    multiconsole scraper fastscraper
    37
    221
    240613
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • paradadf
      paradadf Staff @wggallagher last edited by

      @wggallagher what does the console says after you press ENTER on the appending questions?

      If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

      wggallagher 1 Reply Last reply Reply Quote 0
      • wggallagher
        wggallagher @wggallagher last edited by

        @wggallagher 64 bit computer

        wggallagher 1 Reply Last reply Reply Quote 0
        • wggallagher
          wggallagher @wggallagher last edited by

          @wggallagher nothing. It just goes off screen. Like it closes

          paradadf 1 Reply Last reply Reply Quote 0
          • paradadf
            paradadf Staff @wggallagher last edited by

            @wggallagher what happens if you type a specific folder?

            If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

            wggallagher 1 Reply Last reply Reply Quote 0
            • wggallagher
              wggallagher @paradadf last edited by

              @paradadf if I run fast scraper as admin, it says I dont have scraper.exe, even though I do, and asks if if like to go to the website to get it

              1 Reply Last reply Reply Quote 0
              • wggallagher
                wggallagher @paradadf last edited by

                @paradadf if I tell it to scrape nes, instead of all, it does the same thing after the append question. But I can just run scraper.exe and it starts downloading pictures into a "images" folder it creates

                paradadf 1 Reply Last reply Reply Quote 0
                • paradadf
                  paradadf Staff @wggallagher last edited by

                  @wggallagher have you changed anything inside the batch file? Try downloading it again. I have no idea what is happening. Otherwise, it must be something wirh your computers configuration.

                  If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                  wggallagher 1 Reply Last reply Reply Quote 1
                  • wggallagher
                    wggallagher @paradadf last edited by

                    @paradadf no, I didn't touch the batch file. I'll try downloading again. I see that it's suppose to produce a images folder and a gameslist.txt. Where do the they go on the memory card, if I get this to work? I know I have to stop emulationstston. I just don't know where to put those 2 new items though when I get them

                    paradadf 1 Reply Last reply Reply Quote 0
                    • paradadf
                      paradadf Staff @wggallagher last edited by

                      @wggallagher They go exactly as the structure you created on your computer (that should be the same as on Recalbox). Next to the roms goes the gamelist.xml and the subdirectory downloaded_images.

                      If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                      wggallagher 2 Replies Last reply Reply Quote 1
                      • wggallagher
                        wggallagher @paradadf last edited by

                        @paradadf ok, the 2 items created go in the roms folder. OK, thank you very much.

                        1 Reply Last reply Reply Quote 0
                        • wggallagher
                          wggallagher @paradadf last edited by

                          @paradadf I redownloaded and same results. Guess im SOL, thanks for your help anyway.

                          1 Reply Last reply Reply Quote 0
                          • goyetus
                            goyetus last edited by paradadf

                            Im getting "too many connections" error the first time I use the FastScrapper.
                            I dont know what to do...... any suggest?
                            Thanks!!

                            Scraping atari2600 in progress. Please wait...
                            
                            2016/12/06 03:55:01 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping atari7800 in progress. Please wait...
                            
                            2016/12/06 03:55:01 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping atarist in progress. Please wait...
                            
                            2016/12/06 03:55:02 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping cavestory in progress. Please wait...
                            
                            2016/12/06 03:55:02 INFO: Checking for new hash.csv.
                            2016/12/06 03:55:02 INFO: hash.csv "e2315a44c3d4c5d200477a109e079f29" up to date.
                            
                            
                            Scraping fba in progress. Please wait...
                            
                            2016/12/06 03:55:02 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping fba_libretro in progress. Please wait...
                            
                            2016/12/06 03:55:03 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping fds in progress. Please wait...
                            
                            2016/12/06 03:55:03 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping gamegear in progress. Please wait...
                            
                            2016/12/06 03:55:03 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping gb in progress. Please wait...
                            
                            2016/12/06 03:55:04 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping gba in progress. Please wait...
                            
                            2016/12/06 03:55:04 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping gbc in progress. Please wait...
                            
                            2016/12/06 03:55:05 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping gw in progress. Please wait...
                            
                            2016/12/06 03:55:05 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping lutro in progress. Please wait...
                            
                            2016/12/06 03:55:05 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping lynx in progress. Please wait...
                            
                            2016/12/06 03:55:06 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping mame in progress. Please wait...
                            
                            2016/12/06 03:55:06 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping mastersystem in progress. Please wait...
                            
                            2016/12/06 03:55:06 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping megadrive in progress. Please wait...
                            
                            2016/12/06 03:55:07 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping moonlight in progress. Please wait...
                            
                            2016/12/06 03:55:07 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping msx in progress. Please wait...
                            
                            2016/12/06 03:55:07 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping msx1 in progress. Please wait...
                            
                            2016/12/06 03:55:08 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping msx2 in progress. Please wait...
                            
                            2016/12/06 03:55:08 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping n64 in progress. Please wait...
                            
                            2016/12/06 03:55:09 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping neogeo in progress. Please wait...
                            
                            2016/12/06 03:55:09 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping nes in progress. Please wait...
                            
                            2016/12/06 03:55:09 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping ngp in progress. Please wait...
                            
                            2016/12/06 03:55:10 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping ngpc in progress. Please wait...
                            
                            2016/12/06 03:55:10 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping o2em in progress. Please wait...
                            
                            2016/12/06 03:55:10 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping pcengine in progress. Please wait...
                            
                            2016/12/06 03:55:11 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping pcenginecd in progress. Please wait...
                            
                            2016/12/06 03:55:11 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping prboom in progress. Please wait...
                            
                            2016/12/06 03:55:11 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping psx in progress. Please wait...
                            
                            2016/12/06 03:55:12 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping scummvm in progress. Please wait...
                            
                            2016/12/06 03:55:12 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping sega32x in progress. Please wait...
                            
                            2016/12/06 03:55:13 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping segacd in progress. Please wait...
                            
                            2016/12/06 03:55:13 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping sg1000 in progress. Please wait...
                            
                            2016/12/06 03:55:13 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping snes in progress. Please wait...
                            
                            2016/12/06 03:55:14 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping supergrafx in progress. Please wait...
                            
                            2016/12/06 03:55:14 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping vectrex in progress. Please wait...
                            
                            2016/12/06 03:55:14 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping virtualboy in progress. Please wait...
                            
                            2016/12/06 03:55:15 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping wswan in progress. Please wait...
                            
                            2016/12/06 03:55:15 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping wswanc in progress. Please wait...
                            
                            2016/12/06 03:55:15 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping zx81 in progress. Please wait...
                            
                            2016/12/06 03:55:16 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            
                            Scraping zxspectrum in progress. Please wait...
                            
                            2016/12/06 03:55:16 INFO: Checking for new hash.csv.
                            got 429 Too Many Requests response
                            
                            Start    :  3:55:00,90
                            Finish   :  3:55:16,88
                                     ---------------
                            Duration : 00:00:15,98
                            
                            D:\0_COPIA_SEG_RECALBOX\recalbox\roms>
                            
                            1 Reply Last reply Reply Quote 0
                            • paradadf
                              paradadf Staff last edited by paradadf

                              I'd say you are having too many requests response XD
                              That depends on the data source. Have you increased the workers? Probably your IP is doing too many requests in a short period of time and the API is blocking you, I don't know.

                              Fastscraper ist just a sort-of interface for sselph's scraper. I don't know exactly what every error displayed by it means, sorry.

                              If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                              goyetus 1 Reply Last reply Reply Quote 1
                              • goyetus
                                goyetus @paradadf last edited by

                                @paradadf

                                It Only happens at night. At mid-day is perfect, so I suppose is a server limit for Ip range.


                                Creo que hablas Español, asi que tambien te lo dejo escrito en español 😛
                                Creo que solo ocurre de noche, a medio dia funciona perfecto. creo que es una limitacion que esta aplicando el servidor a un rango de ips en concreto (yo estoy con movistar)

                                1 Reply Last reply Reply Quote 0
                                • pyrostic
                                  pyrostic last edited by pyrostic

                                  @paradadf
                                  I'm pretty NooB at this. How do I run commands from the RP3 in order to stop the emulation station before copying? I'm using a Windows PC.

                                  paradadf 1 Reply Last reply Reply Quote 0
                                  • paradadf
                                    paradadf Staff @pyrostic last edited by

                                    @pyrostic i strongly recommend you to read the wiki/manuals, as all the basic (and some advanced) festures and tricks are explained there.
                                    SSH:
                                    https://github.com/recalbox/recalbox-os/wiki/Root-access-on-terminal-(EN)
                                    WinSCP:
                                    https://github.com/recalbox/recalbox-os/wiki/Network-access-with-WinSCP-(EN)

                                    If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                                    1 Reply Last reply Reply Quote 0
                                    • Keladir
                                      Keladir last edited by

                                      Ok. I got the scraper to scrape my ROM files and create the xml file and images folder. When I put my SD card in and open the main folder, all I have is a boot folder of around 50 MB (of a 32 GB card) that doesn't show any of my ROM files to be able to place the images folder and xml file in there. When I put the SD card into my raspberry pi zero and load up Retropie, all my ROMS are on there and I'm able to play them, but it doesn't show any of the image files or game data if I put the folder and xml onto the boot area I'm getting when I plug it into my PC.

                                      What am I doing wrong? Are the ROMS on a separate partition, or something?

                                      paradadf 1 Reply Last reply Reply Quote 0
                                      • paradadf
                                        paradadf Staff @Keladir last edited by

                                        @Keladir first of all, you are on the recalbox forum. for retropie specific stuff, you should use its forum.

                                        Now about your "issue"... Windows PCs can't read the extra partitions, so you need special software to access them. And as clear as I was on my instructions on the first post, you need to place both, gamelist.xml and downloaded_images folder next to your roms. So I just don't get why would you place it anywhere else. I doesn't make any sense.

                                        If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                                        Keladir 1 Reply Last reply Reply Quote 0
                                        • Keladir
                                          Keladir @paradadf last edited by

                                          @paradadf I opened it up on Retropie's file explorer to poke around, and when I imported my ROMS via USB thumb drive, it copied over the images folder and xml file into the same directory as all the roms, "next to" them as you say, yet they're still not showing up when I load into a console's game list.

                                          Your initial post doesn't say where on the SD card to transfer the XML and images folder to.

                                          paradadf 1 Reply Last reply Reply Quote 0
                                          • paradadf
                                            paradadf Staff @Keladir last edited by

                                            @Keladir again, this is the recalbox forum, if you have problems transfering the files, make your question on the retropie forum.

                                            Your initial post doesn't say where on the SD card to transfer the XML and images folder to.

                                            Next to your roms is a unique location. Wherever the correct place were you copied your roms for retropie, that's the place! I don't know how retropie works, but my initial post also states that you must kill emulationstation before trying to edit/copy/transfer/touch the gamelist.xml. And that is valid for emulationstation in general.

                                            Sorry but either you don't want to understand, or you are not paying attention to what you read.

                                            If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post

                                            Want to support us ?

                                            72
                                            Online

                                            99.6k
                                            Users

                                            28.1k
                                            Topics

                                            187.1k
                                            Posts

                                            Copyright © 2021 recalbox.com