I previously had intel 540 10Gb RJ45 cards in both my DS1821 synology and my Epyc-based XCP-ng host.
I recently upgraded to 710-based SFP+ cards (due to power and noise issues).
After the upgrade I cannot get the storage to direct connect between the NAS and the XCP-ng host.
I even thought I did something wrong in the config and installed XCP-ng from scratch with the new network card, but still cannot ping either device.
The two devices are connected directly via a DAC cable.
The network MTU is manually set to 9000 on both sides.
The network on the NAS side is configured as a static IP 192.168.200.2/255.255.255.0 and 192.168.200.4/255.255.255.0 on the server side. Yet I cannot ping either side from the other.
This is actually a broader problem than I realized.
I am also having issues with the connection of the X710-DA2 card to the switch as well.
Initially it didn’t want to grab DHCP, so I set it as static.
After a little while the DHCP now works, and it can be pinged at the interface address from the rest of the network.
However, if I use this interface as a parent for a VM, the VM will fail to aquire a DHCP address as well.
Interestingly I have a 2nd one of these cards that is being used in the NAS, and it seems to work just fine.
Both cards are dell branded Y5M7N 5N7Y5 cards with the X710-DA2 chip.
Thanks Tom - I did that but still getting destination unreachable.
Also, on the port used for LAN connection the MTU was always set to default 1500 and it never worked well.
I’ve ordered an OEM X710-DA2 to see if that makes a difference.
I was so excited to see nearly 40W of power savings from just replacing these 2 cards, and the excitement has certainly faded now…
After replacement of the card everything works as expected.
Still not sure whether the problem was the fact that it was a Dell card rather than OEM Intel or the card itself had an issue, and I’m hesitant to pull a working card out of another machine to test.
Have you heard of any issues with the Dell-branded X710 cards in XCP-ng?
P.S. On the upside, this upgrade resulted in a 40W power use reduction in my rack - I replaced 2 of 540-T2 Intel OEM 10GbE cards and was thus also able to eliminate 2 of the Wiitek SFP+ to ethernet transceivers.
Interestingly, the random reads/writes to NFS share on NAS improved. Maybe it’s the latency improvement?
My Supermicro servers have X710 based cards, Supermicro branded as I bought them with the servers. 10Gtek DAC cables to a Mikrotik switch, no problems so far. Mikrotik goes multimode fiber to the main switch stack.
