Thanks for your answers and sorry for answering that late, but I wasn't at home yesterday.
And @Substring also thanks for your appreciation. I'm also a developer and linux admin, so I think I know what the helpers/developers want to know Thanks!
Okay, in the GUI the n64 controller doesn't appear at all. Only when I push a button the screen "3 controllers detected. Press any key to configure...." appears (3 because of 1 ps2, mayflash with 2 n64 ports). But I shoudn't configure n64 controller in the GUI because after that no rom starts.
So in controller-config, there's only the ps3 controller. So I cant't test the hint of @acris .
But in shell, n64-controllers are recognized and working.
Ok, I just wanted to try your hint @Substring , and realized teh following: I rebooted with just the n64 controller. When I tried to navigate in es homescreen the controller-config-popup appears. I can't skip it since the n64 has no hotkey configured. A usb keyboard (pressing esc) also doesn't help. So when I tried to press some buttons on the n64, in the fade-in-animation of the controller name, I saw, that the first characted is one like utf8_encoding problem. It is a sqaure-character in first char like so:
(three gamepads because I tried first with only the n64, couldn't skip the configure-popup and then rebooted and tried it with an usb keyboard).
So I think there may be a name-mismatch between OS-Name and ES-recognition-Name so that es thinks, there is no config. May be a controller fault...?
Any ideas?
In the meantime I play BanjooKazooie with ps3-gamepad. Also works fine - more or less!
Thanks!