Controller Config All Messed Up

[SF Lee]

Raspberry Pi model: 3
Recalbox version (build) :
EMULATION STATION V2.0.1A
Hardware (Micro SD, Power Supply, etc.): 32GB 2APwr
Controller(s): Joystick with 8 Play button and 2 Start Select button
Currently on 2 piece of separate Double USB Encoder CY-2121 (Reason why i got 2 piece is because i cant get all 20 buttons to recognise as 2 input with 1 Double USB board so i got 2 and they recognised as 2 inputs of 10 buttons each

Problem : i have tried several build 32gb images and even stock retropie and recalbox images but the problem persist and i am really at my wits ends. I can setup and configure both controller properly with the front end Config Controllers and set it up as Gamepad 1 and Gamepad 2 and joystick move up down left right and all button works properly in menu for both players.

once i enter any emu games either one of the game pad will lose most of its button config and that particular gamepad will invert its joystick also i move left it goes right, i move right it goes left. if i exit to main menu after going into a game the config will be messed up.

if i were to setup both controller in Configure Controller once again it will work and if i proceed to reboot or shut down system upon booting up either one of the controller will be messed up again. i cant get the 2 controller 20 buttons to stay configured at all

playing 2 player games is impossible, only 1 side would be playable with all buttons working, the other side will have all button disabled and joystick up and down is correct but left and right is jumbled up.

i tried doing the individual controller setup in each emu and set save on exit it doesnt work also. no matter what i do. been figuring it out for almost 1 week. can anyone please help me?

i have tried to find the config file in ssh but i can only find config for settings like hdmi / save and such but no buttons