I finally found out that you can't choose players order in N64 games. As soon as you know it, fine.
It's getting more complicated when you use USB adapters. I have a 2-ports Mayflash adapter with a single Wii Classic controller and a 4-ports Mayflash adapter with 3 Gamecube controllers. All players always get assigned to the same port on the adapters, even if the port is empty or not. So unless I get a controller for every port, I can't play with multiple players.
Had anyone encountered the same issues ? Any way to fix this ?