I had this problem too with a NESPi case (original one without '+') and a 2.5A power supply.
So I replaced the power supply by one with 3A - no change.
A direct connection of the power supply to the raspberry board resolved the issue so it was clear that the culprit is the NESPi case.
After some googling I added additional connections from the USB port to the on/off switch board and from the switch board to the raspberry board connector. This fixed the issue completely.
Because the existing connections were difficult to remove I decided to add additional ones in parallel. For the connection to the raspberry board, I added plug inserts and connected the parallel cables to other +5V and ground lines of the 40 pin connector.
It's about 20-30 minutes work if you're capable of soldering. Better than throwing away the case. Because the case - besides being crappy inside - looks very nice 
