Recalbox Forum

    • Register
    • Login
    • Search
    • Recent
    • Tags
    • recalbox.com
    • Gitlab repository
    • Documentation
    • Discord

    [BATCH] Scrape your roms on your PC - fastscraper

    Community projects
    multiconsole scraper fastscraper
    37
    221
    240638
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • saulob
      saulob last edited by saulob

      I tried the bat yesterday. It worked. It's fast! That was amazing.

      I copied the new gamelist file (replaced the old one) and the downloaded_images folder. Replaced everything. When I tried it on recalbox nothing was changed. Everything was the old one, including the images. What's wrong?

      I restarted it, and same thing. I executed the 'update game list' and still the same. What I'm doing wrong?

      Thanks 🙂

      Edit: Sorry for the late reply. I will try stoping the emulationstation before trying again. Thanks

      paradadf 1 Reply Last reply Reply Quote 0
      • paradadf
        paradadf Staff @saulob last edited by

        @saulob yes, ES must be stopped before trying to modify the gamelist.xml

        If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

        saulob 1 Reply Last reply Reply Quote 0
        • saulob
          saulob @paradadf last edited by

          @paradadf thanks. I did and it worked 😉

          But what about the game versions? I mean, check the attached image, the tmnt game has differences but it's all the same name. How can It showing the right version?

          like: Game title (World 4 players) , Game title (2 players version) , something like that

          alt text

          paradadf 1 Reply Last reply Reply Quote 0
          • paradadf
            paradadf Staff @saulob last edited by

            @saulob for console games, I believe by default the batch shows all extras in the game's name. With arcade roms, if you are having the behaviour showed in the picture, try opening the batch an changing the flags for that. Everything should be explained there.

            If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

            saulob 1 Reply Last reply Reply Quote 0
            • saulob
              saulob @paradadf last edited by

              @paradadf It's kinda hard.Found nothing about the names.

              I fixed using another scrap, mamescraper. Worked good on the names, perfect everything detailed.

              1 Reply Last reply Reply Quote 0
              • elvis56nl
                elvis56nl last edited by elvis56nl

                I recently started started using emulationstation and recalbox on the pi 3 and fastscraper seems to be the best scraping option by far. I had some trouble getting fastscraper to work because of the system language detection but i was able to remove that part from the script and now all seems to be fine.

                I was thinking of this feature that could be useful I think: Besides the current function of adding -image and -marquee download all available images for a rom and add descriptors to the filenames accordingly. For instance something like this:

                Aladdin (USA)-title.png
                Aladdin (USA)-ingame.png
                Aladdin (USA)-boxfront.png
                Aladdin (USA)-boxback.png
                Aladdin (USA)-box3d.png
                Aladdin (USA)-cart.png
                Aladdin (USA)-marquee.png
                Aladdin (USA)-wheel.png
                Aladdin (USA)-flyer.png
                Aladdin (USA)-mix2.png
                Aladdin (USA)-mix3.png
                Aladdin (USA)-mix4.png
                Would something like this be possible or does it have to be added to the scraper itself?

                paradadf 1 Reply Last reply Reply Quote 0
                • paradadf
                  paradadf Staff @elvis56nl last edited by

                  @elvis56nl

                  Would something like this be possible or does it have to be added to the scraper itself?

                  That's something sselph would need to add to his scraper.

                  If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                  elvis56nl 1 Reply Last reply Reply Quote 0
                  • elvis56nl
                    elvis56nl @paradadf last edited by

                    @paradadf
                    With some creative thinking i was able to make the script do what I had in mind. I decided I wanted my images in subfolders without suffixes as this makes the roms and media work in both emulation station and attract mode easily.

                    I ended up making several batchfiles: fastscraper_box2d.bat, fastscraper_box3d.bat, fastscraper_snap.bat, fastscraper_mix3.bat which all downloaded the specified image type into subfolders. After running all 4 batch files I end up with all available images in the romdir sorted into the subfolders box2d, box3d, mix3, snap and video. I feel this is the best way to store roms and media files as it is self contained and can easily be moved from one installation to another.

                    paradadf 1 Reply Last reply Reply Quote 0
                    • paradadf
                      paradadf Staff @elvis56nl last edited by

                      @elvis56nl good you found the script useful.
                      Anyway, currently you can't download every single media type from screenscraper with sselph's scraper, as you wanted.

                      If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                      1 Reply Last reply Reply Quote 0
                      • elvis56nl
                        elvis56nl last edited by

                        Messing around with the batch file I came across something strange, when console image type is set to mix3 and on running the script you select all it downloads the 2d cover images. When selecting a specific system when running the script it downloads the mix3 images correctly.

                        I'm not sure if this is because of something I changed in the script as I can't run any of the fastscraper scripts released in the last 8 months or so unmodified. I always get an error on detecting system language.

                        paradadf 1 Reply Last reply Reply Quote 0
                        • paradadf
                          paradadf Staff @elvis56nl last edited by

                          @elvis56nl the script isn't guilty of that... it happens when I scrape too, sometimes.

                          About the language recognition... you could force it by placing the right parameter in the right place. Otherwise I'd need some logs to see where the problem is.

                          If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                          1 Reply Last reply Reply Quote 0
                          • Anonpebbles
                            Anonpebbles last edited by

                            Hello, this is a minor issue but I'm currently trying to scrape my dreamcast games, however it doesnt seem to recognise the "dreamcast" folder. I have tried renaming it to dc, segadreamcast and dreamcast but no results. This text file appears in the folder after attempting scraping:
                            Game,Error,Hash,Extra dreamcast\scraper.exe,hash not found,,

                            Any help would be greatly appreciated! Thanks fellas

                            Rpi 3 wew

                            paradadf 1 Reply Last reply Reply Quote 0
                            • paradadf
                              paradadf Staff @Anonpebbles last edited by

                              @anonpebbles the scraper doesn’t scrape the folder but whatever rom file you have inside of it.

                              If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                              1 Reply Last reply Reply Quote 0
                              • Javier Sandoval
                                Javier Sandoval last edited by

                                great exellent tool, thanks alot, i just had a problem with the batch getting stuck as it couldnt recognise the OS lenguage, i deleted that line of code, and after that it worked nice. Ive seen that other dude had a similar problem and thats how i solved mine, so perhaps you might want to look at that part of the code, im using windows 7x64 with sp1 in spanish mexico type. Thanks again.

                                paradadf RandySlim76 2 Replies Last reply Reply Quote 0
                                • paradadf
                                  paradadf Staff @Javier Sandoval last edited by paradadf

                                  @javier-sandoval si fueras tan amable de abrir un issue en github con informacion mas detallada del error, podria echarle un vistazo para intentar evitar eso.

                                  If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                                  Javier Sandoval 1 Reply Last reply Reply Quote 0
                                  • Javier Sandoval
                                    Javier Sandoval @paradadf last edited by

                                    @paradadf said in [BATCH] Scrape your roms on your PC - fastscraper:

                                    @javier-sandoval si fueras tan amable de abrir un issue en github con informacion mas detallada del error, podria echarle un vistazo para intentar evitar eso.

                                    claro que si,gracias

                                    1 Reply Last reply Reply Quote 0
                                    • IceLancer
                                      IceLancer last edited by IceLancer

                                      Hey!
                                      I have constant crash when attempting to scrape
                                      Every single time it will crash on 5 specific games in my mame game list.
                                      Here is data

                                      2017/11/08 00:10:18 arcade srcs:
                                      2017/11/08 00:10:18 console srcs: ss,gdb,ovgdb
                                      2017/11/08 00:10:18 INFO: Checking for new hash.csv.
                                      2017/11/08 00:10:19 INFO: hash.csv "d9eedf6e622a8ab6a1a14486bb90ffb8" up to date.
                                      2017/11/08 00:10:20 INFO: Checking for new OpenVGDB.
                                      2017/11/08 00:10:20 INFO: Upgrading OpenGDB: -> "bc42a41ec8f627815ea1dee9dfcd9514".
                                      2017/11/08 00:10:22 INFO: Upgrade Complete.
                                      2017/11/08 00:10:22 INFO: Starting: test\columns.zip
                                      panic: runtime error: invalid memory address or nil pointer dereference
                                      [signal 0xc0000005 code=0x0 addr=0x28 pc=0x709c45]

                                      goroutine 65 [running]:
                                      github.com/sselph/scraper/rom.getImage(0x9470c0, 0xc042047d80, 0x0, 0x0, 0xc04293a1e0, 0x16, 0x190, 0x0, 0x0, 0x0)
                                      /home/sselph/go/src/github.com/sselph/scraper/rom/rom.go:366 +0xf5
                                      github.com/sselph/scraper/rom.(*ROM).XML(0xc042e2ba80, 0x9470c0, 0xc042047d80, 0xc042049400, 0x6, 0x8, 0xc04203fb10)
                                      /home/sselph/go/src/github.com/sselph/scraper/rom/rom.go:495 +0x62d
                                      main.worker(0x9470c0, 0xc042047d80, 0xc042940180, 0x6, 0x8, 0xc042049400, 0xc04203fb10, 0xc04301a480, 0xc04301a4e0, 0xc042280a80)
                                      /home/sselph/go/src/github.com/sselph/scraper/scraper.go:140 +0x4e8
                                      created by main.crawlROMs
                                      /home/sselph/go/src/github.com/sselph/scraper/scraper.go:223 +0x66b

                                      Start : 0:10:18.64
                                      Finish : 0:10:23.40
                                      ---------------
                                      Duration : 00:00:04,76

                                      There is nothing in the folder except one single game. Columns.zip
                                      And as i said regular scrape without ur batch file will go throu that without error.
                                      Do u know whats causing this?

                                      Ok i went and tested each line separately it seems that -download_marquees="true" is causing the crash.. i cant tell why..


                                      ok Sorry it has nothing to do with ur batch, i have posted issue on sselph git

                                      paradadf 1 Reply Last reply Reply Quote 0
                                      • paradadf
                                        paradadf Staff @IceLancer last edited by

                                        @icelancer yep, exactly. That's something sselph needs to look into.

                                        If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                                        1 Reply Last reply Reply Quote 0
                                        • IceLancer
                                          IceLancer last edited by

                                          Add an option to turn off scraper.exe autoupdate. its easy to compile from git with go. And bat keeps bringing old exe instead of my new compiled one. Had to manually cut the code for update.

                                          paradadf 1 Reply Last reply Reply Quote 0
                                          • paradadf
                                            paradadf Staff @IceLancer last edited by

                                            @icelancer have you tried increasing scraper.exe version number before compiling? I‘d need to check that. If you can, please open an issue about this on github do I don‘t forget about it, please.

                                            If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                                            IceLancer 2 Replies Last reply Reply Quote 0
                                            • First post
                                              Last post

                                            Want to support us ?

                                            86
                                            Online

                                            99.6k
                                            Users

                                            28.1k
                                            Topics

                                            187.1k
                                            Posts

                                            Copyright © 2021 recalbox.com