Great news all!
We were able to make it work finally. So, when using eMMC, the XU4 boot from a "hidden" partition which is not overwritten when flashing a Recalbox image nor upgrading.
You must flash data to that partition manually.
Here are the commands to do that : https://wiki.recalbox.com/en/tutorials/video/tft/second-minitft-or-ogst-for-scraps-and-videos
Thanks to @karl0ss who sent me his eMMC and @cedric-debuiselle for testing.