My network is set up like Bridged router > Unifi USG > Unifi 24 port POE switch
Into the switch I then connected the USG, my Dell PowerEdge R210 II server, workstation, and all the other stuff.
I got 500/500 megabit fiber internet, but I started having a problem where I could only get it up to 100/80 megabit, and it felt very spongy when browsing.
I then started to disconnect things from the switch, and when I got to the R210, internet speed went up to 550/550 megabit. Connecting the server again, and it went down to 100/80 again. I then tried to just power it off, and leaving it plugged in. My internet was then measured to 130/100 megabit.
Pulling the power cable out of the server, and I got 500/500, putting the power cable back in, without powering the server up, and it is down to 130/100 again… Testing the network internally gives me 970 megabit in every configuration… I don’t get it.
Checking with Wireshark, network isn’t getting spammed with broadcasts.
It might be time to swap out the NIC on the R210. They are cheap enough on eBay.
It might also be bad auto negotiation. You might have to manually set the NIC speed to 1Gb.
It is running at 1g, and when testing I am also getting 970 megabit through. The problem isn’t between the server and switch. It feels like it is between the usg and fiber box… But that makes no sense.
Your orginal troubleshooting seems to indicate the R210 is the issue.
Is there anything running of the server that might be consuming bandwidth? Torrents? Try booting a linux distro from USB to see if this changes the behavour.
Seeing as it’s misbehaving you could try putting it on it’s own VLAN to segregate the LAN traffic.
It’s not uncommon for a bad NIC on a device to bring a whole LAN segment to it’s knees.
Most servers will have lights out functionality. The NIC gets power just by plugging in the power cable. The NIC is ON even though the sever isn’t running. That’s why you got those results. The NIC is bad so as soon as it touches the network everything comes to a halt. Check the state of both NICs if they are both bad then get a NIC to put in the spare slot on the R210. If it’s just one then disable it in the BIOS and use the other one.
So I tried to add another NIC… and it works as expected… until it starts the OS up.
Selecting to chose boot device, it continues to boot up as normal, TOTALLY DISABLES THE NIC I ADDED, then asks for what device to boot from. And from that point the NIC is like it is gone from the system.
… starting to migrate over to my R610… I dont know what else to do
It could easily be a bad join in the R210. Opening it up to install the NIC may have improved the bad connection enough for it to work again.
RE the OS not seeing the new NIC that’s down to BIOS settings, the OS drivers and compatibility of the NIC with the OS. This is usually the case although it sounds like you have other issues.
When you are trouble shooting make a single change then test and retest to confirm the results. I can’t count the number of times a simple restart has fixed issues. When a large number of changes are made it’s hard to discover the root issue.
Is the 4 port NIC a known working part? Does it work in other systems?
Is the NIC compatible with the OS?
Does the BIOS see the new hardware? Does the OS see the hardware? Are the drivers being loaded correctly? Are the right drivers being loaded (version etc)? etc, etc. Keep moving up the stack.
Let us know how you go.
The card goes dead just before the OS is even selected.
After running a few hours, the problem came back again.
I have only changed one thing at a time too. I tested the NIC in my R610 before putting it in the R210, and all 4 ports work as expected.
But I still don’t understand how it should be able to mess up the connection between the modem and my USG. That is on the WAN side of the USG. Everything on the LAN side work exactly as it should.
My guess is that the R210 is flooding pause frames to the switch. Since the switch is managed, turn off flow control on the port towards the R210 to see if that is the case.
If the router is sending more broadcasts/multicasts than the other machines on the network (quite normal), the switch is more likely to send the router pause frames (since the switch is supposed to forward the broadcast out of the port towards the R210, and that port is full because the R210 spams pause frames), which in turn will cause the router to reduce its ethernet throughput, and give the ‘only out of the house’ effect you describe.
The switch will not forward pause frames, so you cannot sniff them on other switchports.
To sniff the pause frames, you need a PC with two network cards in bridge mode, inserted in the link between the R210 and the switch.
My first home adventure with this was a TV tuner on my network multicasting more than 10Mbps. My IP phone was 10Mbps, so the switch buffer towards the phone ran full, and the switch started sending pause frames to the tuner. 2 seconds of playback, 2 seconds of frozen image, repeat. Unpluged the phone and all was fine. Now the real fix in my case was set up IGMP properly, but that is not for this discussion.
