Thinking about a 10Gb network? This might help!

How is working out for ya several months later? Are you happy with the results?

I picked up a chelsio T420 cause my other 10Gbe NICs are Chelsio so They can have a Chelsio party LAN. I was NOT successful in getting RJ45 ports to talk to DAC ports on the CRS305 mikrotik switch (something about media conversion I assume cause I tried modules from no less than 5 vendors, including microtik themselves) so I am waiting for the T420 so I can get everything set up with fiber instead of copper.

2 Likes

hi

My freenas 10GB interfaces are down …not sure why.

I have followed your guide and set the system tunables and mtu

i use the same HP cards as you are…HP NC523SFP

i am connecting to the Proxmox server with Freenas with DAC cables…the Proxmox side the interfaces are up and working but the freenas once are down… i tried various things but in vain…need some help.

many thanks

rxx

1 Like

Hi drowsy

many thanks for sharing this… i recently bought 6 of these HP NC523SFP 10GB PCIe dual port server adapters.

these are working fine in my proxmox and freenas set up very well.

I just needed to change the loader.conf file in freenas by adding the line if_qlxgb_load=“YES”

and add the if_qlxgb.ko driver to the kernal-debug folder… otherwise it gives errors while booting to the freenas complaining that no such file was found. I used filezilla to upload the .ko driver file.

system tunable was also added…as mentioned in your post.

shutdown the freenas server…to restart…there you go …qlo and ql1 show up … and no errors.

hope this helps someone.

2 Likes

i have problem with this card “QLogic cLOM8214” when using static ip in Freenas 11.3-U1, but when using dhcp the card works well, have not change anything in system-tunables.

1 Like

I think there is a bug in FreeNAS 11.3U1 with static IPs. Its been reported – sorry I had the page at one time but now for some reason lost the link at jira systems. Anyway the fix is in for U2. Hopefully that will be dropped soon.

1 Like

I had considered creating a franken-switch from a custom built PC to enable 40gb or faster switching in the house through fiber (this was before I knew Cat 8 existed).
So I did a bunch of diving and found multi-port SFP NiCS that would handle in excess of 40gigabit per a port (or so the thing claimed) with the intent of stacking a bunch into a compact case. NiC prices were 450-850 each.
It wouldn’t have been cheap at all but dramatically less than the ultra rare multi-thousand dollar core switches I saw.
I suppose someone will ask why I would do that, and before anyone thinks to ask i’ll just simply say because I wanted to and it sounded like a fun project. I evaluated the cost and compared it to my cashflow, that caused me to toss out the idea but it was a fun mental excersize. From planning stage to budget estimates.

2 Likes

You are the man. Thank you so much for writing this up. I already have similar parts from Chelsio and you’ve inspired me to try getting it all hooked up this weekend! This may also let me sidestep my iSCSI/VLAN problem. I’ll come back and report. :partying_face:

1 Like

Hi ya, great guide so i got me some hp sfp+ dual ports card off ebay for 25$ each and on of then even came with 2 3m long sfp+ cable but my on issue if that i have 2 pcs for testing purposes linked together with file shares between them on of them is a old 4-5th gen i3 with a intel 256Gb ssd and the other my 8th gen i5 gaming rig and for the life of me can’t figure out why on my gaming rig, after setting up the drives and testing the link windows just disables my card so i have to turn my pc off then remove the card power it back up uninstall the driver then turn off replace card turn on reinstall driver then the same thing happens over and over again, im desperate at this point and don’t feel like buying another card for my pc when the 2 i have work fine. (ive tested both on my older test pc and they work fine. Any help at all appreciated!!

1 Like

I’m assuming you are running Windows and you might want to check this setting on your NIC.

image

1 Like

Yeah thats the first thing i tried that fixed the same issue on the older test pc but not mine :frowning:

1 Like

I know this thread is old, but Drowsy’s guide has been immensely helpful to me. I am quite new to TrueNAS let alone 10Gb networking, but so far this year I obtained an H3C 5800 24 port switch with four (4) SFP+ ports, then in August I purchased an HP DL160 G6 server whose onboard SAS controller is not on the TrueNAS list, so I bought a HP P410 Smart Array card that is. Installed that card then TrueNAS on the bare metal. Got a 2-pack of the HP NC523SFP, put one into the TrueNAS box, it recognized it perfectly and works with both a fibre and DAC connection to the switch. I Could not be more happy with things as they are. Will add one card to a Windows Server 2019, in hopes it installs the correct driver, then lastly my Windows 10 workstation, fingers crossed for this 3-way setup!

2 Likes

Does anyone have a link for the Windows 10 drivers that @drowsy posted? The QLogic Windows Server Drivers link: Support | Public Driver Downloads | QLogic Fibre Channel HBAs and FastLinQ Ethernet adapters and controllers - Marvell
Instead now shows this and says the driver downloads are only for IBM customers:

1 Like

Quick update: FWIW I did find an Amazon question with some links to HP downloads:

However, hard to know if there’s more updated drivers available from QLogic directly. If there are, if anyone has those drivers they could upload somewhere that’d be awesome.

Resulting driver version from the HP link:
image

Update: (Had to make a new post since new users can only have two external links per post).
The QLE3242 drivers are still available for download here and I confirmed the firmware and the driver versions match what’s available from HP.

I’m rather new to SFP+ cards, but I’ve had 4 of these NC523SFP cards for over a year but haven’t been able to use them. I’ve been using a few single port Intel X520 cards that were just plug and play but I’d prefer having the Dual port HP cards usable. I’ve extracted the drivers from the HPE link but have no idea how to install them and my system doesn’t even detect the card in device manager.

I’m running Windows 10 on my main rig, and Manjaro and POPos on my other PCs. I’ve even endured the Arduous adventure of even doing a Ryzen Hackintosh with Opencore.

Is there a guide or thread you could direct me to for how to install those drivers or update the firmware?

I’ve got a few PCs on hand and a TrueNAS box and spare drives I could install whatever OS on to update the firmware. Kinda at a loss for where to begin.