@Basti1993
Update:
Also occurs on another Raspi4 with a vanilla Recalbox installation. So it is a general issue.
When i use an older Xbox Wireless Adapter (the bigger one) it works for 30 min and then the controllers disconnect. I can reproduce that behaviour. The pairing button on the Wireless Adapter also doesnt work after that (LED is on but seems to be frozen). Only a reboot solves it. USB Cable works all the time.

Thanks 🙂