XCP-NG 10gb Question/Issue

Computer Hardware & Server Infrastructure Builds
1

What type of performance should be expected when a Xen VM pulls a file from a physical server? I have two identical HP servers. One running Hyper-V and the other XCP-NG 8.3. Everything is 10gb DAC.

Hyper-V VM consistently pulls the file around the 200MB/sec mark
XCP-NG VM consistently pulls the file around 110MB/sec mark

The XCP-NG performance stays the same regardless of xen NIC type, jumbo frames on/off, and Citrix vs Vates drivers.

I did however notice that if I have multiple VMs on XCP-NG pulling that file they can both pull at over 110MB/sec at same time. Which lead me to believe there is a 10GB pipe there, but per VM or per stream seems to be limited to 1gb speeds.

Are XCP-NG VMs limited to 1Gbps speeds per stream/per vm?

2

Depends on the protocols that you used. SMB, NFS?

I can confirm that it will push 10Gb on VM’s. I’ve used the xen project drivers. The NICs will show 100Gb link after installing the drivers. Have you tried direct connect from server to server?

3

I tried both S3 and HTTP. From a Hyper-V VM I’m getting 270MB/sec. My S3 setup is MinIO and for http its apache hosting a 10GB iso file as a test.

My Xen VM does show 100Gbps as the link. Xen VM to Xen VM I get around 300MB/sec. Xen VM to Physical I’m limited to 110MB/sec or so.

4

In general, I don’t think your XCP speeds are far off from my testing. I’m hoping for a big jump when version 9 gets released, but it is still going to be a while. Based on Alma 10.

1 Like
5

I have this working in production right now. Nothing special at all. 10G links and my VMs saturate the links. Not sure where in your architecture is having the bottleneck.

6

What is your physical and virtual network config? I’ve noticed that if you use a 10GbE as your management interface, and then setup a virtual network on that same NIC for VMs, then the network throughput of those VMs is limited - I can’t remember of they were limited to 1Gbps speeds (110-125MB/sec), but I do remember that they were limited. If my memory is working, this is by design on the management interface, to prevent any unintentional impact on the critical management interface, because this interface is used for Xen Orchestra comms, and by default for pool traffic, backups, VM migrations if not using shared storage, etc.

Good practice is to have separate, dedicated NICs for management, storage, and VM network traffic. If you have the hardware (server and network), here’s the official guidance: XCP-ng Networking

7

I did have management on the 10gb bond (active-backup) on all hosts. I ended up having to move mgmt to a dedicated NIC because mgmt kept dropping pings no matter what I did. When I did that it fixed ping issue and did seem faster. I’ll do that to the rest. Thanks!!!

1 Like