I've found a fix, in case someone needs a help with this issue as well.
Load a game, then press Select +B to bring up the Retroarch menu.
From there, press A and go to Configurations > Audio.
Now set the audio volume and mixer volume to -10.
Hope it helps.