Slow 10GB Network + Unifi Gen2 + FreeNas

Networking & Firewalls
1

Hey Everyone!!

I am new to 10GB game but I am getting extremely slow speeds on my 10GB network, 135-170 MB/sec using iperf3, between my desktop computer and my FreeNas box. I just added the 10GB cards: (1) Chelsio T520-CR in my FreeNAS Mini running FreeNAS-11.3-U4.1 and (2) Intel X540-T2 running the lastest intel driver 4.1.197.0 on a home built Desktop computer running Windows 10 Build 1904.

I am using the SPF+ ports on my UniFi Gen2 switch to connect the devices.

Any help would be greatly appreciated!!

Thanks

2

If the NICs use the same type of ports, I would connect the systems directly to each other and see if it could be an issue with the switch. If you have two ports on a NIC you could also IP them on the same network and connect them to each other and run your test. This should help isolate which of the three is the issue.

1 Like
3

The desktop has rj45 ports and the FreeNAS box has SPF+ SR fiber ports. Therefore, I like your idea of checking the connections between ports on each card which would allow me to also check the wiring in the walls. What would be the best way to IP them together and testing software to use?

Jerry

4

On the Windows machine, you will need to setup a VM and have it use one of the ports. The base OS can use the other and just connect a cable between them. I would run iPerf3.

On the FreeNAS, I would create two VMs and have them each use a dedicated port to test between them.

5

Have you set the MTU settings to 9000 on both interfaces?

6

Thanks for all the advice!!

Just a quick update: As long as I mainly set network IP addresses on my 10 GB devices that are different from my main network (i.e., 10.1.1.x vs. 192.168.20.x), I am able to get 9.4+ GB/s throughput while directly connected to my devices as well as running everything thru my switch. However, as soon as I enable DHCP on my 10 GB devices and add them to my main network, the speed drops to ~1.6 GB/s which indicates to me that all my network traffic is moving thru my pfsense router (SG-4860). Therefore, is there a way to configure my pfsense router or UniFi Gen 2 switch to handle all the local traffic on the switch without trans-versing the router?

Thanks for helping a 10GB Newbie

7

Anytime you are routing traffic between subnets, I would expect it to be much slower than 10Gbit unless you have a high end router such as a Cisco ASR. You need to have both devices on the same network so traffic between them is only switched and not routed.