Welp, I don't know how I did it, but I managed to fix the issue. it really was the recalbox-user-config.txt. I made sure to delete all my userscripts, pasted in this code in the user config: dtoverlay=gpio-fan,gpiopin=14,temp=60000 and connected my resistor wire to the corresponding GPIO pin and now it works exactly how I want.
I started noticing the fan was actually starting and stopping while the box was on after moving everything to a new SD card. I don't think it was the card that fixed it as I copied everything in both the recalbox and share drives to the new image, overwriting everything with my configs. The only difference is I removed the bash and python scripts I was experimenting with in the userscripts folder. The user-config still had the previous line of code and actually started working.
I still had one problem as I was still using GPIO 3 instead of 14, and the fan would stay on if I quit recalbox through the menu instead of a hard power down by holding down the button. Switching to GPIO 14 like I originally wanted and updating the code to match the pin fixed this, thankfully. Now I have a properly functioning mini fan that came with the case.
Tl;Dr;
If you buy a Geekpi case for Raspberry Pi 5, buy a EZ fan2 for it with some extra wires. Install the 5v and ground wires into pins 4 and 6 like the Geekpi instructions say, and add your new resistor wire to pin 8 (GPIO 14).
654d39e7-7853-4b1a-bb9a-1af511f4bd58-image.png
Set up your recalbox as recommended, doing the first time boot and setup so it creates the needed files.
Open up your recalbox-user-config.txt located in the root of the RECALBOX drive either on your computer or via a network.
Paste in dtoverlay=gpio-fan,gpiopin=14,temp=60000 in a free line.
#Add a comment line above the code like this to write down it's the fan control code if you'd like
reboot
And that should work. I will experiment to see if why it didn't work for me before might have been because I wrote 55000 for the temp and that was too low or not, but I'm satisfied with 60 degrees as a trigger.
I will also quickly mention that the EZ fan2 does fit in the rather compact case. Just tuck it under the board with the wires in an "S" shape.
geekpi wiring.jpg
I still think it would be nice if there could be a GPIO fan control menu in emulationstation like Raspberry Pi OS has, but I managed to work around it.
Thanks for listening and hope this helps someone in the future.