XCP-NG and Intel X520-DA2 10GBit card with non-Intel SFP+ transceivers

Computer Hardware & Server Infrastructure Builds
1

Hi Guys,

I recently did setup a new XCP-NG host, which has an Intel X520-DA2 10GBit card in it.
Transceivers are from Finisar.

The ixgbe driver in its current configuration does not recognzie these non-Intel SFPs:

[00:56 vmcluster04 ~]# dmesg | grep SFP
[   11.367930] ixgbe 0000:03:00.0: failed to load because an unsupported SFP+ or QSFP module type was detected.
[   11.400061] ixgbe 0000:03:00.1: failed to load because an unsupported SFP+ or QSFP module type was detected.
[00:56 vmcluster04 ~]# #

I know, that I can allow non-Intel SFPs by adding a kernel option “allow_unsupported_sfp=1”, however: I did not figure out on how to do that on the XCP-NG environment.

Trying to remove the module and insert it again with the parameter passed to the command failed also:

[00:56 vmcluster04 ~]# rmmod ixgbe
[01:00 vmcluster04 ~]# insmod /usr/lib/modules/4.19.0+1/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko allow_unsupported_sfp=1
insmod: ERROR: could not insert module /usr/lib/modules/4.19.0+1/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko: Unknown symbol in module
[01:00 vmcluster04 ~]#

I would be happy if someone could advise?

Thanks a lot!

2

Follow-Up:

I tried to add this to /etc/modprobe.d/ixgbe-options.conf:

options ixgbe allow_unsupported_sfp=1

and after doing a:

rmmod ixgbe
modprobe ixgbe

I could find this in the dmesg log:

[14:44 vmcluster04 ~]# dmesg
[  641.481979] Intel(R) 10GbE PCI Express Linux Network Driver - version 5.5.2
[  641.481981] Copyright(c) 1999 - 2018 Intel Corporation.
[  641.482211] xen: registering gsi 30 triggering 0 polarity 1
[  641.482228] Already setup the GSI :30
[  641.500233] ixgbe: 0000:03:00.0: ixgbe_check_options: FCoE Offload feature enabled
[  641.500235] ixgbe: allow_unsupported_sfp Enabled
[  641.511343] ixgbe 0000:03:00.0: failed to load because an unsupported SFP+ or QSFP module type was detected.
[  641.511361] ixgbe 0000:03:00.0: Reload the driver after installing a supported module.
[  641.511760] xen: registering gsi 37 triggering 0 polarity 1
[  641.511776] Already setup the GSI :37
[  641.532226] ixgbe: 0000:03:00.1: ixgbe_check_options: FCoE Offload feature enabled
[  641.532227] ixgbe: allow_unsupported_sfp Enabled
[  641.543354] ixgbe 0000:03:00.1 0000:03:00.1 (uninitialized): WARNING: Intel (R) Network Connections are quality tested using Intel (R) Ethernet Optics. Using untested modules is not support                                                                                                                             ed and may cause unstable operation or damage to the module or the adapter. Intel Corporation is not responsible for any harm caused by using untested modules.
[  641.681819] ixgbe 0000:03:00.1: Multiqueue Enabled: Rx Queue count = 16, Tx Queue count = 16 XDP Queue count = 0
[  641.684051] ixgbe 0000:03:00.1: 32.000 Gb/s available PCIe bandwidth (5 GT/s x8 link)
[  641.684394] ixgbe 0000:03:00.1 eth2: MAC: 2, PHY: 17, SFP+: 6, PBA No: G28774-000
[  641.684397] ixgbe 0000:03:00.1: a0:36:9f:27:9f:0a
[  641.684414] ixgbe 0000:03:00.1 eth2: Enabled Features: RxQ: 16 TxQ: 16 FdirHash
[  641.686499] ixgbe 0000:03:00.1 eth2: Intel(R) 10 Gigabit Network Connection
[  641.731152] ixgbe 0000:03:00.1 side-4034-eth2: renamed from eth2
[14:44 vmcluster04 ~]#

So, the option itself results at least in a change of behaviour, however, only one of the two 10G ports seems to be detected on PCI slot 0000:03:00.1 - the other one at 0000:03:00.0 is allegedly still equipped with an unsupported SFP module - but it’s the same one as in the other port.

Also, according to the dmesg log, one of these interfaces has been added as eth2, but when trying to acquire this interface:

[14:50 vmcluster04 ~]# ifconfig eth2
eth2: error fetching interface information: Device not found" 
[14:50 vmcluster04 ~]#

I have no further ideas at this stage. Any help would be appreciared!

Thanks!

3

I assume you have tried restarting the server…

4

Of course :slight_smile:

But none of the above things is reboot-persistent so nothing of that will survive a reboot

5

ixgbe.allow_unsupported_sfp=1,1 is the correct kernel boot option for activating both SPF ports

I have one system that uses a legacy bios and one server with EFI (both dell R420 servers)

so far I have just modified the /boot/grub/grub.cfg file to add ixgbe.allow_unsupported_sfp=1,1

if you have a different way to keep it persistent would be great :wink:

6

tl;dr

edit:
nano /etc/grub.cfg - Legacy BIOS
nano /etc/grub-efi.cfg - UEFI BIOS
to include the ixgbe.allow_unsupported_sfp=1,1 parameter on the first module2 line (at the end).

FWIW, anyone having this issue in the future.

X570D4U, 5950X, 64GB, 2x1TB 970Pro, SMCSE-835TQC-R802B, RTX-A4000, X520-DA2 (Intel 82599) with V2tech SFP-10G-LR-X-AV2 module (CISCO Compatible, not intel).

I have both 10G ports working on the X520, omitting the , between the 1’s causes the array to only find and use only the last NIC of a multi-port adapter.

RaZer0r was helpful in pointing to the right journey starting place, but the config is more nuanced that those simple statements for those of us not fluent in kernel parameters :wink:

I found an example for adding to the grub file here: Troubleshooting | XCP-ng documentation

I also used this guide to help troubleshoot the ixgbe driver: Installing and updating the ixgbe driver – IT Blog