[BATCH] Scrape your roms on your PC - fastscraper

[DavidinCT]

@paradadf I'm just a Windows user, so thanks for your help again. After moding your script to fit my needs. I didn't even know what a Recallbox was till this morning. Sorry about that.

The machine is a HTPC, and for the roms, I dropped in a SSD so I get performance on them, they are in the local machine. By just dropping things in the ROM folder, it saves me a hassle anyway, as my roms are NOT on the drive. I just dont like how I see the desktop when loading a rom from ES...sigh, nothing I can do about that one.

I did need to edit the XML for it to work (it's nice too, as I can edit the name MUCH easier than in ES),

Going through all the NES Roms for example with the built in thing (with 46 roms), it would take hours and time out 1/2 the time, so about 50% of them had no data (once it times out, it's done till you do a new search). With this script, it found 99% of them (a few not found ones but, those I can do manually or find a new rom) and only took about 45 seconds...

Loving it... In my tool box for life...

[paradadf]

@DavidinCT I only created the script, but the credit for the scraper goes to sselph.
Also, take a look for the Universal XML Scraper if you don't know it yet ;).

[DavidinCT]

@paradadf said in [BATCH] Scrape your roms on your PC - fastscraper:

@DavidinCT I only created the script, but the credit for the scraper goes to sselph.
Also, take a look for the Universal XML Scraper if you don't know it yet ;).

What is it and what does it do ? I'm using Notepad++ (Notepad on steroids one of my fav apps)

Thanks again for your help !

[paradadf]

@DavidinCT just do a simple search on this forum and find it out.

[levendis32]

@paradadf Hello I have move my ROMS to PC but I cant find the .bat file to download it from github.The link I have is this https://github.com/sselph/scraper/releases but where is the bat file I dont know. I know that this is a 1st class NEWBE question but please show me the obvious because I m STUCK

[paradadf]

@levendis32 you should always read the very first entry on any topic. That's where usually the instructions are

Just right click and download as...
https://raw.githubusercontent.com/paradadf/recaltools/master/fastscraper/fastscraper.bat

fastscraper automatically downloads sselph's scraper.

[gmgman]

I like your script, it's pretty effective. I have one request, for game naming: is it possible to extract game names from their file, without having the bracket stuff and the file ending in it? e.g. if my SNES rom is called "Ys V - Kefin, Lost Kingdom of Sand (J) [T+Eng 1.00 Aeon Genesis].zip" your script either make the game names like the file itself or takes the name from the Database which will turn into "Ys V: Ushinawareta Suna no Miyako Kefin", I'm not happy with either of those and would simply prefer "Ys V - Kefin, Lost Kingdom of Sand", substring from file name before the first bracket. maybe you can add an option to set the game name stripped from the filename without the stuff in the brackets and without the .zip etc. ending? just a suggestion...

[gmgman]

@paradadf looks like your script doesnt add file endings already, but I would prefer to not see the stuff in (...) and [...] in the game name

[paradadf]

@gmgman the name convention doesn't come from my script but from the scraper itself. But I can try to add that option for you. I'm pretty busy right now though, so it might take me a while.
The only problem I see is that you might end up with many duplicates if you don't clean up your romset properly.
I'll let you know when the feature is implemented.

[gmgman]

@paradadf thanks for the quick response! yeah, I try to have my rom collections clean (1g1r) so I don't worry about dupies. dupies also appear if i get the game names from the database, e.g. Illusion of Gaia (USA) and Illusion of Time (Europe) both end up as Illusion of Gaia if I use the DB naming option.

Thanks to everyone involved in the recal box project. this is so much better than retropie!

[gmgman]

@paradadf. I was just thinking, maybe it's wiser to keep square brackets [...] in the game name, or maybe let the user set which brackets they want to remove from the game name?

[paradadf]

@gmgman hahaha I don't know if ai can do that easily in a batch script, but I'll try.
Maybe it would be just easier to use notepad++ to delete specific strings all at once... lets see.

[gmgman]

@paradadf the most part which I find annoying in the game name is the country codes and the language codes (En,Fr,Es) etc.. sure I could use notepad++ and search for strings to replace in the gamelist.xml file but then there are so many different combinations depenting on the rom set and I would have to make sure to only edit where <name>..</name> tags are and not remove them in the filename tags etc. which will make it more complicated. well it's all just an idea, no need to rush on this idea tho, I stick to the file name for now because I rather see the titles as I named the rom files instead of what the DB offers.

[fullskull]

Holis...eem...yo er...tengo un problemo con el fastscraper. Yeske, lo pongo en la ruta de las roms y cuando ejecuto el .bat desde windows, salta una ventana de cmd y se cierra al segundo de abrirse. ¿Que puede estar pasando?

[paradadf]

@fullskull probablemente el contenido del batch que descargaste no sea el correcto. Abre el archivo con algun editor de texto, copia y pega el contenido RAW de github.

[fullskull]

@paradadf ya hice de nuevo el archivo bat y ahora si parece hacer otra cosa, pero la otra cosa que hace es mostrarme un error.
[img]http://i.imgur.com/0vH8AkK.jpg[/img]

[paradadf]

@fullskull tu instalacion de windows debe ser minima y le faltaran algunos datos del registro. De todas formas deberia ponerse en ingles y funcionar sin problemas.

[fullskull]

@paradadf si lo hago desde macintosh, ¿crees que me podria ir mejor? tambien tengo linux ubuntu pero no se como hacer para ejecutarlo desde ahi.

[paradadf]

@fullskull si utilizas la version bash, funciona desde mac y linux. En la carpeta de github te dice como hacerlo. Primero le das permisos de escritura con chmod y luego lo ejecutas.
De todas formas, siempre y cuando tengas powershell instalado en tu version de windows, podes utilizar esa version sin problemas.

[fullskull]

@paradadf ,men, no doy con ello, no se como arrancarlo desde mac y es que en el enlace de descarga tampoco dan mucha instrucción, tal vez si, pero quizá para los mas entendidos. Quizá un tutoría con el paso a paso de como hacerlo funcionar estaría bastante bien. Luego no se porque, al darme por vencido con el fast scraper e intentado probar el universal XML scraper y aunque me ha descargado todo los snapshots, me sigue mostrando la ruta de los juegos como una lista, sin añadir su foto correspondiente.