I use OVH bare metal as well, but instead use ESXi with a virtual pfSense as my firewall. First thing is make sure you create an ACL for access limiting only traffic from your IP to the mgmt port for XCP-ng.
Once you spin up the VM, then you need to console in and assign the initial IPs. I also spun up a Windows box so I could access the firewall via the web interface to complete the config.
The default gateway that I had to configure on the firewall took some guessing since I was assigned a /32. I used the same gateway address as my ESXi mgmt port and just changed the subnet mask to a /16. Traffic started flowing fine with that config.