Well, I managed to fix this issue. The fix is odd and probably specific to Dell machines but go into the BIOS and go to General -> Advanced Boot options, Untick Enable Legacy option ROMS

Machine now boots fine with joystick connected....