Recalbox Forum

    • Register
    • Login
    • Search
    • Recent
    • Tags
    • recalbox.com
    • Gitlab repository
    • Documentation
    • Discord

    [BATCH] Scrape your roms on your PC - fastscraper

    Community projects
    multiconsole scraper fastscraper
    37
    221
    240603
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • paradadf
      paradadf Staff @Anonpebbles last edited by

      @anonpebbles the scraper doesn’t scrape the folder but whatever rom file you have inside of it.

      If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

      1 Reply Last reply Reply Quote 0
      • Javier Sandoval
        Javier Sandoval last edited by

        great exellent tool, thanks alot, i just had a problem with the batch getting stuck as it couldnt recognise the OS lenguage, i deleted that line of code, and after that it worked nice. Ive seen that other dude had a similar problem and thats how i solved mine, so perhaps you might want to look at that part of the code, im using windows 7x64 with sp1 in spanish mexico type. Thanks again.

        paradadf RandySlim76 2 Replies Last reply Reply Quote 0
        • paradadf
          paradadf Staff @Javier Sandoval last edited by paradadf

          @javier-sandoval si fueras tan amable de abrir un issue en github con informacion mas detallada del error, podria echarle un vistazo para intentar evitar eso.

          If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

          Javier Sandoval 1 Reply Last reply Reply Quote 0
          • Javier Sandoval
            Javier Sandoval @paradadf last edited by

            @paradadf said in [BATCH] Scrape your roms on your PC - fastscraper:

            @javier-sandoval si fueras tan amable de abrir un issue en github con informacion mas detallada del error, podria echarle un vistazo para intentar evitar eso.

            claro que si,gracias

            1 Reply Last reply Reply Quote 0
            • IceLancer
              IceLancer last edited by IceLancer

              Hey!
              I have constant crash when attempting to scrape
              Every single time it will crash on 5 specific games in my mame game list.
              Here is data

              2017/11/08 00:10:18 arcade srcs:
              2017/11/08 00:10:18 console srcs: ss,gdb,ovgdb
              2017/11/08 00:10:18 INFO: Checking for new hash.csv.
              2017/11/08 00:10:19 INFO: hash.csv "d9eedf6e622a8ab6a1a14486bb90ffb8" up to date.
              2017/11/08 00:10:20 INFO: Checking for new OpenVGDB.
              2017/11/08 00:10:20 INFO: Upgrading OpenGDB: -> "bc42a41ec8f627815ea1dee9dfcd9514".
              2017/11/08 00:10:22 INFO: Upgrade Complete.
              2017/11/08 00:10:22 INFO: Starting: test\columns.zip
              panic: runtime error: invalid memory address or nil pointer dereference
              [signal 0xc0000005 code=0x0 addr=0x28 pc=0x709c45]

              goroutine 65 [running]:
              github.com/sselph/scraper/rom.getImage(0x9470c0, 0xc042047d80, 0x0, 0x0, 0xc04293a1e0, 0x16, 0x190, 0x0, 0x0, 0x0)
              /home/sselph/go/src/github.com/sselph/scraper/rom/rom.go:366 +0xf5
              github.com/sselph/scraper/rom.(*ROM).XML(0xc042e2ba80, 0x9470c0, 0xc042047d80, 0xc042049400, 0x6, 0x8, 0xc04203fb10)
              /home/sselph/go/src/github.com/sselph/scraper/rom/rom.go:495 +0x62d
              main.worker(0x9470c0, 0xc042047d80, 0xc042940180, 0x6, 0x8, 0xc042049400, 0xc04203fb10, 0xc04301a480, 0xc04301a4e0, 0xc042280a80)
              /home/sselph/go/src/github.com/sselph/scraper/scraper.go:140 +0x4e8
              created by main.crawlROMs
              /home/sselph/go/src/github.com/sselph/scraper/scraper.go:223 +0x66b

              Start : 0:10:18.64
              Finish : 0:10:23.40
              ---------------
              Duration : 00:00:04,76

              There is nothing in the folder except one single game. Columns.zip
              And as i said regular scrape without ur batch file will go throu that without error.
              Do u know whats causing this?

              Ok i went and tested each line separately it seems that -download_marquees="true" is causing the crash.. i cant tell why..


              ok Sorry it has nothing to do with ur batch, i have posted issue on sselph git

              paradadf 1 Reply Last reply Reply Quote 0
              • paradadf
                paradadf Staff @IceLancer last edited by

                @icelancer yep, exactly. That's something sselph needs to look into.

                If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                1 Reply Last reply Reply Quote 0
                • IceLancer
                  IceLancer last edited by

                  Add an option to turn off scraper.exe autoupdate. its easy to compile from git with go. And bat keeps bringing old exe instead of my new compiled one. Had to manually cut the code for update.

                  paradadf 1 Reply Last reply Reply Quote 0
                  • paradadf
                    paradadf Staff @IceLancer last edited by

                    @icelancer have you tried increasing scraper.exe version number before compiling? I‘d need to check that. If you can, please open an issue about this on github do I don‘t forget about it, please.

                    If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                    IceLancer 2 Replies Last reply Reply Quote 0
                    • IceLancer
                      IceLancer @paradadf last edited by IceLancer

                      @paradadf Should i have? I just compiled from git

                      BTW , answer me this if u can , thanks.
                      Is it possible to tell batch when asking for system name , to give him folder name with space ?
                      I am in process of sorting rom folder to their original names and i am stuck with this bat now.

                      i tried :
                      Sega Dreamcast
                      "Sega Dreamcast"
                      'Sega Dreamcast'
                      it keeps finding Sega OR Dreamcast, as not found. But not Sega Dreamcast

                      IceLancer 1 Reply Last reply Reply Quote 0
                      • IceLancer
                        IceLancer @IceLancer last edited by

                        @icelancer meh 😞 not even "all" can see folders with spaces..
                        for /F "delims=" %f in ('dir /B /A:D') do set system=!system! %f
                        Lists folders properly:
                        set system=!system! Sega Dreamcast

                        Why isnt it finding Sega Dreamcast then?

                        paradadf 1 Reply Last reply Reply Quote 0
                        • IceLancer
                          IceLancer @paradadf last edited by

                          This post is deleted!
                          1 Reply Last reply Reply Quote 0
                          • paradadf
                            paradadf Staff @IceLancer last edited by

                            @icelancer EmulationStation doesn‘t have system names containing more than one word (or spaces), so I didn‘t bother taking that case into account. Do you really need it? Why don‘t you use folder names like ES?

                            About your first question... could you send me your compiled version so I can test it?
                            Maybe changing == by GEQ in the update check might be enough.

                            If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                            IceLancer 1 Reply Last reply Reply Quote 0
                            • IceLancer
                              IceLancer @paradadf last edited by IceLancer

                              @paradadf Sorry but yeah, i am kinda heavy on heaving everything sorted top notch. Making rom folder names to their specs is priority to me. I already spent a week modifing all database to match it.
                              it also makes all 3 frontend systems working in sync
                              BUT its not priority for you at all. I found workaround for batch fastscraper by doing dirty hack to it.

                              here is scraper

                              https://mega.nz/#!MxQ0HJQa!h6RU6cHjV83F8T7d7wPDOfFJdwo_P8t0EKz-ruVDa4c

                              it reads :
                              fastscraper ver. 14.09.2017

                              paradadf 2 Replies Last reply Reply Quote 0
                              • paradadf
                                paradadf Staff @IceLancer last edited by

                                @icelancer I‘ll see what I can do. Hopefully I find some time to test one thing before sselph uploads a newer version 🙂
                                I meant the scrapers version, not fastscrapers version. But don‘t worry... your compiled version should be enough.

                                If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                                1 Reply Last reply Reply Quote 1
                                • paradadf
                                  paradadf Staff @IceLancer last edited by

                                  @icelancer there is something weird with your compiled version. When running a batch with the command scraper.exe -version you should get something like v1.4.4 and not an empty string. Moreover, your scraper.exe is over 8 mb while the one sselph offers is only 5,5 mb. I´m not sure where in his code the versioning is being made, but you´ll need at least the same version number as the latest one available on github for the update script to be skipped. Erasing that part of the code is easier (if you really need it) instead of adding a switch.

                                  About your folder names with spaces... I´ll look into it when I have time.

                                  If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                                  IceLancer 1 Reply Last reply Reply Quote 0
                                  • IceLancer
                                    IceLancer @paradadf last edited by IceLancer

                                    @paradadf I don't know why the discrepancy in size. Its clean Git compile.
                                    I tried scraper -version
                                    doesnt print anything.
                                    Btw i noticed its not just me who is insisting on up to the code rom folder names. Every Hyperspin user insists on the same as well, i didnt knew that. But when i think more on it i honestly do not see any legitimate reason for ES to use manufactured names like they do.Its pure chaos and a mess.

                                    P.S
                                    I also noticed havoc when collections got support in ES. People even made topic to organize "new unified" names for them. Its hilarious.

                                    1 Reply Last reply Reply Quote 0
                                    • RandySlim76
                                      RandySlim76 @Javier Sandoval last edited by RandySlim76

                                      @javier-sandoval Can I inquire what exactly was changed in the fastscraper.bat file to fix the Windows 7 64bit issue you were having?
                                      I am having the exact same issues. Thank you.

                                      Also if an Admin or if anyone knows how or what he did to get fastscraper.bat to get past this registry error I would deeply appreciate it. Thank you.
                                      0_1516803873096_IMG_20180124_082423.jpg

                                      paradadf 1 Reply Last reply Reply Quote 0
                                      • paradadf
                                        paradadf Staff @RandySlim76 last edited by

                                        @randyslim76 for the interface language, I added this line which might help you if the pc language isn‘t recognized:
                                        https://github.com/paradadf/recaltools/blob/master/fastscraper/fastscraper.bat#L11

                                        If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                                        RandySlim76 1 Reply Last reply Reply Quote 0
                                        • RandySlim76
                                          RandySlim76 @paradadf last edited by RandySlim76

                                          @paradadf I changed the language to en on mine and the language error is gone. I'm proud you figured that part out. Now the registry error from what I read above when you helped out the chap to figure out his 64bit issues he said he deleted and or edited several lines of the code. Do you have any idea what these two gentlemen did to make it work without the error? Thanks in advance. 0_1516838330062_IMG_20180124_175839.jpg

                                          Also I haven't had a chance to try it but would this be the section that is causing the registry error?
                                          0_1516838417656_IMG_20180124_175457.jpg

                                          These are the changes he said he made to get the batch working.
                                          0_1516839110562_IMG_20180122_115805.jpg

                                          paradadf 1 Reply Last reply Reply Quote 0
                                          • paradadf
                                            paradadf Staff @RandySlim76 last edited by paradadf

                                            @randyslim76 probably... if you can‘t automatically overcome that point, try setting/forcing the arch variable one line after the code you just posted.
                                            If afterwards the download, extraction, etc isn‘t working, just download the corresponding version from sselph scraper github page and place the scraper.exe nect to the fastscraper.bat.

                                            If your question was answered, please mark it as solved: Topic Tools -> Ask a question + Topic Tools -> Mark as Solved

                                            RandySlim76 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post

                                            Want to support us ?

                                            72
                                            Online

                                            99.6k
                                            Users

                                            28.1k
                                            Topics

                                            187.1k
                                            Posts

                                            Copyright © 2021 recalbox.com