Not quite finished but this config might be of help for anyone stumbling upon this topic:
After adding n64.configfile=dummy to recalbox.conf to avoid configgen to generate the button mapping, you can copy this configuration into /recalbox/share/system/configs/mupen64/InputAutoCfg.ini:
[USB GamePad USB GamePad] plugged = True plugin = 2 AnalogDeadzone = 2096,2096 AnalogPeak = 21912,21912 Mempak switch = Rumblepak switch = mouse = False A Button = button(1) B Button = button(2) C Button D = axis(2+) C Button L = axis(3+) C Button R = axis(3-) C Button U = axis(2-) DPad D = hat(0 Down) DPad L = hat(0 Left) DPad R = hat(0 Right) DPad U = hat(0 Up) L Trig = button(6) R Trig = button(7) Start = button(8) Z Trig = button(9) X Axis = axis(0-,0+) Y Axis = axis(1-,1+)I need to dig deeper into the AnalogDeadZone and AnalogPeak to find the best way to get the proper values.
Do not forget to delete the garbage characters on /recalbox/share/system/.emulationstation/es_input.cfg
(.emulationstation is a hidden directory!)
I‘ll share a proper mapping for it as soon as I can.