Poor performance Synology / XCP-NG

Computer Hardware & Server Infrastructure Builds
1

Hey All, I have a Synology SA3200D with 12x Seagate Nytro 3332 in RAID 10 running across a 10Gb isolated network. XCP-NG on a 32 core Epyc CPU with 512G of memory. I allocated 32 cores and 128Gb to a single VM (the first VM - being used for testing). I ran Phoronix on it and am getting poor scores.

I’ve verified that there is ~10Gb of available throughput from the XCP-NG host and Synology box(docker iperf3), I also ran the local benchmark utility per disk on the Synology giving over a Gig of throughput per disk and 200k+ IOPs per disk. RAID 10.

The Phoronix test is here comparing my system against that of a Rackstation running in “raid” F1
is available here. TLDR My system underperforms in 90% of the tests.

What are some things to look at here?

2

My thought would be the CPU being the bottleneck. When you run your performance testing from the VM, what metrics do the Synology show?

3

roughly 15% utilization currently. rerunning the test and watching the resources more closely.

4

Are you using a switch or directly connecting the storage?

5

Its going through a switch currently, but I did try a DAC direct between the 2 boxes thinking network was a likely issue, unfortunately, no go there either.

6

That’s good. It’s all about isolating the issue. I would try to install one of the hard drives locally in the server to see if you get better results. Then you would know it isn’t an issue with the host or VM.

7

The 32 cores, is that processor multi threaded so you are really only using half of the processor? If not you might try backing it down a little, the host needs 4 cores and some of the ram to operate. After that I have no suggestions.

8

It is multi-threaded, I originally started with a modest VM of 4 cores and 8Gb with the same results. figuring that maybe that was the problem I bumped it way up in performance.

I ran the PTS-FIO test in docker directly on the synology and saw a massive difference in random write performance (Results here)

I either have a network problem or something on the local machine. I ran the test against the local raid 1 NVME pair and see similar results (rerunning to post results currently) so it seems to be something with xcp-ng?

I also ran crystal disk mark on a new windows VM and see very similar results of poor random read and write performance

9

After continuing to play around with this, and running driver updates on the window system. I got a significant improvement in performance to where id expect it to be. It seems there is something with our particular install on linux VMs that is preventing storage from being properly utilized.

1 Like