@zing first of all, thank you for your help/time. I appreciate that.
I tried everything you mentioned and ended up trying another OS (don't want to advertise others than Recalbox so no name as long as you don't ask).
I have zero issues with games, although i prefer recalbox in general. I still use recalbox for my arcade machine but will stick to the other OS for the living room, where we mainly use the Pi for KODI.
and a HUGE SURPRISE... the CEC Adapter works fine now with the other OS. I never managed to get my remote working with KODI/Raspberry Pi4/Recalbox. But now, it works out of the box, without any tweaks. just plug and play.
So I am happy with Recalbox in my arcade cabinet and another OS in the living room for KODI.
kind regards.