TrueNas Scale hardware upgrade - NIC not working

Computer Hardware & Server Infrastructure Builds
1

Hello!

So I finally got some new hardware to upgrade my TrueNAS server. Its now on the following..

Motherboard: MSI Pro B550M-VC Wifi
CPU: AMD 3950x
RAM: x2 Crucial Pro RAM 64 GB Kit (2x32GB) DDR4 3200MHz (CP2K32G4DFRA32A) 128GB total
Boot drive: Crucial P310 500GB NVMe
Storage Drives: x2 SEAGATE IRONWOLF PRO 16TB NAS HDD
PSU Corsair RM850x

Aside from some fans and a cooler, That’s all the new stuff. Things i transferred were..

x3 SEAGATE IRONWOLF PRO 16TB NAS HDD (in ZFS1)
x2 Crucial MX500 2TB SSD (used for App Vcash)

The Issue:

While TrueNAS booted up, it does throw some ‘red’ text on the screen while booting up (it goes too fast for me to catch what it is). It does boot up, however it is not using the onboard NIC. It can see it, as much as it knows its there, but doesn’t seem able to use it. I’m guessing its a driver issue.

In the terminal, I pulled up this information.

2a:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15)

Guessing that, since TrueNas isn’t using it, there isn’t much I can do to get it to work. If that is the case, then I guess im in the market for a new NIC. If I can get it to work, is there any guidance?

I see lots of people recommending a 10GB SFP card (such as the INTEL X520-DA2 which is out of production). Problem is, I’ve never messed with SFP cards or ports. I’ve got a (cheap) Treadnet 2.5GB switch, so no SFP ports on it. I know using SFP cards require modules in addition to the cards themselves. Would I be able to use an SFP card on my current switch? Or would I need to get an entirely new switch?

Right now, I have it connected via a USB-Ethernet dongle, but that isn’t ideal.

Thanks in advance.

2

If you don’t have a 10gbe switch, I wouldn’t bother with an SFP+ NIC. If you really want to go 10gbe you could go with an intel X550 or X710 card that has ethernet ports. But really with a 2.5gig switch, why not just go for a 2.5g NIC? You can get an Intel i-226 NIC for under $20

https://www.newegg.com/p/14U-014G-00007?Item=9SIB3EWKFB0214

3

I’d like to eventually upgrade my network, so im not oppose to future proofing things (and truth is my current switch has a dead port). But yeah, at the moment, im just on a 2.5gb switch.

4

As for the SFP+ connection, I’ve also seen a few posts in various places mirroring this reply.

Is there a reason you need to go RJ45 though? Why not use SFP+ and get something like the X710-DA2? When I priced things out, it was way cheaper going with SFP+ ports using twinax cables. Plus, the SFP+ ports are modular so you can just insert optics in the future if needed.

As i said, I’m certainty not opposed to going with either route, and do plan to (eventually) upgrade my network (when space allows) and it would be simpler to just swap out modules when the time comes, than swap out network cards. Plus, it may help me learn more about SFP and how it works and what all it can do.

Is there a reason to avoid it with my current setup, other than I wouldn’t see any speed advantage out of it?

5

SFP+ with Twinax or DAC or AOC only work if you have SFP+ ports on both sides of the connection, which your switch does not. The only way to connect SFP+ to your switch would be if you put 10baseT transceivers into the X710 card, which will cost you approximately $50 each, and may not work at all, depending on the card you buy. Not all NICs will support the 10base T transceivers. Plus the 10baseT transceivers run very hot. Its just a waste (in my opinion) to try and use SFP+ NICs if your switch has no SFP+ ports. The real advantage of SFP+ is being able to use a DAC or an AOC to connect your NIC to the switch. Both a DAC (direct attached cable or twinax) and an AOC (active optic cable) are extremely low power (little or no heat) and extremely low latency. Once you put an ethernet transceiver in the mix, you lose those benefits.

6

You may want to consider upgrading your switch if you are dead set on SFP+. Something like this may be suitable

7

I would go Intel x710 if you want SFP+, it’s a PCIe3.0 device and the x520 is a PCIe2.0 device and may slow things down. The cost difference is around $10-$15 on the used market.

You can pick up an i226 card for around $30 used and upgrade to 10g when it makes sense and you upgrade your switch. Good 10base-T adapters will cost you over $50, just to throttle it down to 2.5g. I have not found cheap and reliable dedicated 2.5g or 5g SFP+modules yet.

8

I woudln’t say im “dead set” on SFP+, but would like to eventually learn about it and such.

I will need to upgrade my switch at some point for sure. Tho ill probably look for something a bit bigger than the one you listed (and i cant say that the lack of information and reviewers complaining about the quality of it gives me confidence in that MikroTik one either)

Chances are good I’ll be able to buy one when a nearby university gets rid of some, so ill keep an eye out for them. (im also sitting on an auctioned Netgear PoE switch, but the PoE doesnt work and it doesn’t have SFP ports, but got it pretty cheap. Would love to figure out how to fix the PoE on it though).

In the meantime, i ordered an x550-T2 card to use with it. Should do for a while, long as i don’t have any issues with it, and i can figure out why my server is randomly locking up :expressionless: (something ill post about elsewhere).

Thanks for all the info :slight_smile:

9

I now have 4 Mikrotik switches, 2 in production, 2 in lab. The lab switches were bought used. No problems with any of them so far, only annoyance is the wall wort on both of the CRS309-1g-8s+in switches. I wish they would ship with a power supply that takes a regular AC cord.