Recalbox Forum

    • Register
    • Login
    • Search
    • Recent
    • Tags
    • recalbox.com
    • Gitlab repository
    • Documentation
    • Discord

    Scraping with PC - Batch process for you to try!

    Recalbox General
    scraping process try
    12
    17
    8178
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • neko
      neko last edited by

      Hi, just managed to get my Raspberry Pi 2 a few days ago, I spend my holiday to rebuild everything from zero, so I come back to share my results. This time, I got around 15000 ROMs on my SDCard, all ran through the scrapev2-deluxe.bat you supplied, done in a few hours without much problems. I must say that the scrapping speed and overall quality is very good, and I am very happy with what I get at this moment.   However I still have a few questions and see if you can help: 1. I noticed that the boot up time increased significantly after the scrape. The loading screen holds for around 2 minutes, is that normal? 2. About the scrapping results of mame and fba games, V1 gives good picture quality but lacks of description. V2 gives good descriptions but the screenshots looks pretty bad. Is there anyway to improve the situation by adjusting the parameters of the batch files? 3. It looks like the scrape.exe does not work with FDS, MSX, SG-1000, MegaCD and PCEngine CD. Does anyone have some luck with these systems?

      1 Reply Last reply Reply Quote 0
      • nickdj
        nickdj last edited by

        Thanks for the feedback.  I will have another look at the scrape results.  The 'scraper.exe' file allows a different parameter for image quality but the description should be there.  I will have a look at it later. I've noticed boot times increase the more roms you have.  Not got as far as 2 minutes but I have about 4000k roms and it takes about 30 seconds now.   I've been working on a new theme for recalbox, I'll be posting a pre-view in a bit.

        RPi 2 Not overclocked Edimax EW-7811UN LogiLink BT0007A USB bluetooth V2.1 PS3 Controller RecalBox v3.2.8

        1 Reply Last reply Reply Quote 0
        • nickdj
          nickdj last edited by

          I'm testing the FBA scraper and for some reason it's not picking up any descriptions.  The descriptions work within Recalbox though.  Tried to use a newer version of the scraper.exe but getting the same results.  😞

          RPi 2 Not overclocked Edimax EW-7811UN LogiLink BT0007A USB bluetooth V2.1 PS3 Controller RecalBox v3.2.8

          1 Reply Last reply Reply Quote 0
          • airchrysalis
            airchrysalis last edited by

            Hi Nick, Did you ever figure out how to manually scrape and get it to save after shutdown? I'm trying to put metadata for folders containing psx games with multiple .iso files but the data is always lost when I shutdown. Do you have any suggestions?

            1 Reply Last reply Reply Quote 0
            • krantz
              krantz last edited by

              To save manually scraped metadata, you have to quit EmulationStation NOT using the quit menu, but pressing F4 on the keyboard.

              1 Reply Last reply Reply Quote 0
              • nickdj
                nickdj last edited by

                I'm struggling with one final issue with the scraper and that's FBA games do not have a description.  I've tried all sorts to pull down this information but it's not working. Has anyone managed to scrape FBA games via Windows Commandline and managed to get a description?

                RPi 2 Not overclocked Edimax EW-7811UN LogiLink BT0007A USB bluetooth V2.1 PS3 Controller RecalBox v3.2.8

                1 Reply Last reply Reply Quote 0
                • rockaddicted
                  rockaddicted last edited by

                  Nope no descriptions about the arcade's games. Certainly a problem from the games's database. We are working on scrapping database to add natively to recalbox in the futur. (take so many time... so be patient ^^) If someone want add the descriptions manually, tell us. We'll add them to the database 😃 https://github.com/digitalLumberjack/recalbox-gamelists (for information, this database is based on the set games no-intro + arcade)

                  Recalbox V4.1.0 - RPI3 OC - Xbox360 wireless
                  Wiki ENG : https://github.com/recalbox/recalbox-os/wiki/Mini-How-To-(EN)
                  Wiki FR : https://github.com/recalbox/recalbox-os/wiki/Mini-How-To-(FR)

                  Don't forget to upvote messages if it has been useful ;)

                  1 Reply Last reply Reply Quote 0
                  • 0zzy
                    0zzy last edited by

                    hi all, i have followed this guide and firt of all this one https://github.com/digitalLumberjack/recalbox-os/wiki/Scrape-your-games-with-your-PC-(EN) but when i reboot the system, xml files are empty and no game details or images are shown in the emulator menu. what's wrong?

                    1 Reply Last reply Reply Quote 0
                    • rberthao
                      rberthao last edited by

                      This process works great, thanks!

                      1 Reply Last reply Reply Quote 0
                      • liquidsnakeblue
                        liquidsnakeblue last edited by

                        Is there any way to modify the batch file to DELETE roms it does not find metadata for?  If a rom's hash cannot be matched to the database it is more than likely a modified rom and I do not want it.  This way after the process is complete only good roms with matching metadata will show up in Recalbox. Liquidsnakeblue

                        1 Reply Last reply Reply Quote 0
                        • randomninjaatk
                          randomninjaatk last edited by

                          I would also be a fan of having the option to have the batch file to possible either move or delete roms that do not match up in the database via the hash. This would allow to have the cleanest looking library and hopefully a very valid set of roms. I just found this tool today and it's amazing! Thanks for all your work on it!

                          1 Reply Last reply Reply Quote 0
                          • deano66
                            deano66 last edited by

                            Hi I'm new to the forums ! I have been playing around with Recalbox over the past few months on and off and think it is awesome 🙂 I have built a number of different arcades using Mala and Hyperspin and hope to create a mini sized machine using Recalbox. I have had no luck with scraping images using all the methods above 😞 I can use the internal scraper and it saves and displays the images but it uses the same image for a number of different roms and is slow. If I use the above methods on a pc I get near all the images and they are accurate and quick but I can not see them on recalbox when I load it up. Sometimes a few systems will show(I have tested it with a fresh install maybe 20 times) and at other times none show or the box were the images should be shows but with nothing in it 😞 any help would be very much appreciated as I maybe doing something wrong but I have followed the tutorial as described and still have the same problem. cheers and thank you in advance

                            1 Reply Last reply Reply Quote 0
                            • darkbrokenheartz
                              darkbrokenheartz last edited by

                              I have a pretty foolproof and simple method that does not require to use te external pc.   Source https://github.com/sselph/scraper     Showing simplified method to quickly scrape artwork for Recalbox     Step 1.  Connect to your raspi.  On windows you can use putty  and just open a connection to your ip and use root for username  recalboxroot for password.   Or from a terminal client >ssh root@yourip   Step 2. Paste the following command > wget https://raw.githubusercontent.com/substring/fullscrape/master/fullscrape.sh && chmod u+x ./fullscrape.sh   Step 3.  Initiate an individual scrape request system by system For example to do SNES  see below > ./fullscrape.sh snes -no_thumb=true -max_width=375   To do nes > ./fullscrape.sh nes -no_thumb=true -max_width=375   Using the rom folder name as reference. Just a list of examples of systems you can use are   cavestory fba fbalibretro fds gamegear gb gba gbc gw lutro lynx mame mastersystem megadrive moonlight msx n64 neogeo ngp pcengine prboom psx scummvm sega32x segacd sg1000 snes vectrex virtualboy wswan     If you want to update or only add images that have not already been scraped you can do   > ./fullscrape.sh -u  -no_thumb=true -max_width=375     Things to keep in mind. To run the ./fullscrape.sh script you must be in the directory that you originally downloaded it to in the first step.   Image art is stored in the directory /root/.emulationstation/downloaded_images/ the gamelist.xml file is stored in the directory /root/.emulationstation/gamelists each in their respective named folder   if you run ./fullscrape.sh without the no thumb width flag it will download unvisible art for certain systems and you will need to delete that before scraping again as it will not overwrite correctly.  (This could be added as a pull request to github but may be a security feature)   WARNING : fba_libretro has a little bug, that will be corrected in 4.0.0 : scraping can’t be displayed. This can be manually solved if you really really are desperate …

                              1 Reply Last reply Reply Quote 0
                              • First post
                                Last post

                              Want to support us ?

                              77
                              Online

                              99.6k
                              Users

                              28.1k
                              Topics

                              187.1k
                              Posts

                              Copyright © 2021 recalbox.com