Hello are you using a CRT screen? you can try to edit your recalbox.conf like that :
## If you use rice core, set mode 480p as below
n64.core=rice
n64.videomode=CEA 2 HDMI
then inside your /recalbox/configs/mupen64/mupen64plus.cfg add your screen resolution here :
[Video-General]
# Use fullscreen mode if True, or windowed mode if False
Fullscreen = False
# Width of output window or fullscreen width
ScreenWidth = 1280
# Height of output window or fullscreen height
ScreenHeight = 720
# If true, activate the SDL_GL_SWAP_CONTROL attribute
VerticalSync = False
In this example, it is setup for a HD resolution of 720p.