I finally got it to work:
- removing the n64.config="dummy" from recalbox.conf (can be skipped when nothing was done before)
- readding input.xml (can be skipped when nothing was done before)
- configure n64 controller in recalbox (a button in menu is b button on controller, b button in menu is a button on controller, all other buttons as expected, skip hotkey)
- setting
/recalbox/share/roms/n64/.retroarch.cfg
to
input_autodetect_enable = true
input_menu_toggle_gamepad_combo = 7
- enter game long hit start button (2 seconds) to enter retroarch quick menu
- hit b
- go to settings -> input -> Port 1 Controls
- configure controller
- hit save controller profile
this will save the controller profile for future starts.
exiting, saving etc. will work when long pressing start to enter retroarch menu.
I hopes this does help somebody with n64 usb controller, recalbox 7.1.1, rpi 4 raspberry 4