@paradadf said in [BATCH] Scrape your roms on your PC - fastscraper:
@DavidinCT I only created the script, but the credit for the scraper goes to sselph.
Also, take a look for the Universal XML Scraper if you don't know it yet ;).
What is it and what does it do ? I'm using Notepad++ (Notepad on steroids one of my fav apps)
Thanks again for your help !
@paradadf I'm just a Windows user, so thanks for your help again. After moding your script to fit my needs. I didn't even know what a Recallbox was till this morning. Sorry about that.
The machine is a HTPC, and for the roms, I dropped in a SSD so I get performance on them, they are in the local machine. By just dropping things in the ROM folder, it saves me a hassle anyway, as my roms are NOT on the 
drive. I just dont like how I see the desktop when loading a rom from ES...sigh, nothing I can do about that one.
I did need to edit the XML for it to work (it's nice too, as I can edit the name MUCH easier than in ES),
Going through all the NES Roms for example with the built in thing (with 46 roms), it would take hours and time out 1/2 the time, so about 50% of them had no data (once it times out, it's done till you do a new search). With this script, it found 99% of them (a few not found ones but, those I can do manually or find a new rom) and only took about 45 seconds...
Loving it... In my tool box for life...
@DavidinCT OK, I think this needed it's own post. I did get it WORKING and working in ES on Windows...paradadf you have been moved to a GOD status in my book !
FIRST CLOSE ES and MAKE SURE it's not running in the task manager.
Ok, When this script runs on Windows, it creates in the folder in the roms folder for each system. For example.
Roms\nes
Roms\nes\downloaded_images (Image store)
Roms\nes\gamelist.xml (game list and meta data)
and
Roms\nes_nes_missing.txt (roms that were not found)
In the Windows version,
There is a few folders where this data is stored. Starting in the root of the <user> folder by default.
C:\users<user name>.emulationstation
C:\users<user name>.emulationstation\roms
C:\users<user name>.emulationstation\downloaded_images
C:\Users<user name>.emulationstation\gamelists
In each of the ROM, Downloaded_images and gamelists, there are sub folders for each system, named accordingly, like nes, snes, atari2600, etc. So they should all line up.
Now looking over the NEW gamelist.xml that is in your ROM folder...here is an example...
<path>./Castlevania 64.n64</path>
<name>Castlevania (USA)</name>
<desc> (SNIPPED FOR EXAMPLE).</desc>
<image>./downloaded_images/Castlevania 64-image.jpg</image>
The problem is, the image store is and as this is n64 game...
C:\users<user name>.emulationstation\downloaded_images\n64
So you would need to change that to, again example
~/.emulationstation/downloaded_images/n64
With notepad++ you can do a easy search and replace. save the file, drop in the correct folder (.emulationstation\gamelists<system name>) and over write the old one, then CUT all the images out of the downloaded_images folder (in the rom folders, and move them to the official store .emulationstation\gamelists<system name>)
Once this is done restart ES and enjoy your new images.
Hope this helped someone !
@paradadf Real system ? I told you this system was designed as a media PC and on Windows 7 (that even updates ARE disabled on as too many problems over the years), I don't even think PowerShell is installed (I believe it's an optional feature in 7). I was not kidding ! 
Note, this system is not used for anything else, Not even a browser is opened on it unless it's for a driver, Even the host file is broken down so every ad site is blocked.
Anyway, the Atari 2600 finished up (250 or so roms), still the XML file and the Images are in the ROM folder, NOT in the \downloaded_images or \gamelists.(there are separate root folders for it)
I tried this before, leaving the XML in the folder with the images but, it didn't do anything or moving the files to the correct directories didn't do anything.
Here is something, I don't know how ES on windows processes information. Can it be manually imported ? What can I do with the xml to get it in the system with the according folder layout ?
sigh, so close...
Edit: finished up, looking over the Atari 2600 games, OMG it found like 99% of them, The built in thing only found about 15 out of 250ish. I really hope I can get this working !!!!
Edit 2: Oh, well.... Ran ES, No data updates keeping the files where they are (as your script created). Anyone else running the Windows version of ES using this and how did they get the data/images working correctly ?
The data and images are there, how can I use them ??
@paradadf Think I got it working. I did try to the set arch and I got stuck on the NEXT part...downloading...and stuck again...sigh
So, I pulled out a LOT of stuff..
-CPU check (changed to amd64)
-Lang check (changed to EN)
-Removed check version, download and extract (that whole section, as it was getting stuck on each part)
Now running it, it asked me the questions and seems to be running now.
Crossing fingers this works... remember my folder layout is not exactly common for this and before nothing would show up in ES...
Trust me, I really hope this all works... If you want me to up the mod that I did, let me know as it could help others with funky problems.
@paradadf Think I found out what PART of my problem was....
I was using notepad...and word wrap was on... when saving it modified the file and caused errors (have not done that in years), Notepad++ only going forward... this gets me backwards tho...
Running as an Admin....
ERROR: The system was unable to find the specified registry key or value.
Can't get OS's language. English will be used.
from this point it just hangs here. Is there anything that needs to be enabled for this script to work ?
This is a HTPC, this is a very stripped down feature machine, only setup for media viewing. That I wanted to use some of it's horsepower for some classic gaming (funny it runs Wii and WiiU games fine too..LOL)
Thanks again for your help, I could understand if you were done at this point...
Under a command line...
reg Query "HKLM\Hardware\Description System\CentralProcessor\0"
ERROR: The system was unable to find the specified registry key or value.
Manually checking that key Closest you get to AMD64 is "Intel64 Family 6 Model 60 Stepping 3"
@paradadf It gets to ...
G:\EmulationStation.emulationstation\roms>reg Query "HKLM\Hardware\Description
System\CentralProcessor\0" | find /I "x86" 1>nul && set arch=386 || set a
rch=amd64
And just hangs....
@paradadf 4.10.2016 is the one that works...17.02.2017 does not work (something I have to play with)
@paradadf Yes, I know a batch file is not too hard. With out looking over it everything in the file, I am not sure. I have one that DOES run that I cant log into but, it is creating XML and images so, what ever on this one...as long as it works, right ?
I've been playing with this for a few hours now. I figured out the folder layout in Windows, and there is a
\downloaded_images
and
\gamelists
Folders with sub folders named for each system and deleing these folders, and re-running ES, they get re-created on the drive, so now I know what ones mean anything.
I copied my 20gb of roms on to the drive (correct folder layout) for this test, so I can rule things out.
When I run the fastscraper, I see it creates a gamelist.XML and a downloaded_images folder in the rom folder. So I have just copying them into the correct folder as they are laid out (overwriting the old XML file).
Problem is once I open Emulation Station, No changes show in there. Even after a reboot.
I figured the Meta data on ES would be the EASY part...in fact it's been the hardest part. Over a week working with itand 1/2 my games don't have meta data.
I was very excited to see another meta data option....just wish it was working for me 
@paradadf I have a Home theater PC that I am using this on, It's based on Windows 7 because it's for TV watching as well (Windows Media Center). So I needed Emulation Station for Windows. Scraping has been a headache so if this works I wanted to use it. And the PC I am using is FAR more powerfull than a Pi is, as it's designed to be able to play 4K video.
I don't know, I set to raw, copy to a txt file and drop in the root of the rom folder, renamed to bat, it does not work. I tried it 2 times
I dropped one that you linked directly above and it seems to work. but, it does not allow me to set my LOGON info for that site.
I hope I am not missing anything by not setting the logon info.
Windows 7 x64
The install is at C:\users<user name>.emulationstation
In the config file, I set the roms and emulators to point to a G:\Emulationstation.Emulationstation
So, emulators and roms are not on the OS drive. Like this it does not work (as images and XML files are on the drive). I copied the ROMS over to the drive (as a temp thing)
I ended up downloading your one that points to screenscraper.fr and it seems to be working, I think. It's giving an error that API not available to NON-MEMBERS. The site is in French and I cant figure it out to setup an account.
I could not find a direct download on
https://github.com/paradadf/recaltools/tree/master/fastscraper
So, I copied the text and created a batch file, pasted it in...maybe some errors (I viewed as raw).
Sigh....
I tried this and so far getting no where.... I have maybe 500 roms... The Emulationstation folder is in the user path (c:)but, the roms are located on another drive(g:), so it's kind of broken up trying to keep the ROMS off the drive. With this the system works perfectly fine (config files are set right). I dropped this batch file in the root of the rom folder.
The only problem when I run this, I just get a message of
Error: The system was unable to find the specified registry key or value, Cant get the PS's language. English will be used.
And a blinking _
it's been sitting there for over an hour like this, never asked me to do anything. Am I doing something wrong ? Or should I be doing something different for my setup ?