@LTS_Tom recommends Intel cards with BSD driver support for all kinds of hardware offloading etc.
The sg2100 includes a hardware 4 port switch…
given a sufficiently equipped host system, could this provide the same combo routing/switching performance as the Netgate appliance?
Depends on your use case. I don’t recommend using the 4 ports as a switch, if you need a switch get a switch.
Bit of a loaded question. After watching your videos I was emboldened to take to pfsense plunge. Repurposed an old i5 with all the crypto, ebay’d an intel quad port, amazon’d a managed switch and a few wireless APs
I figured I could do this:
igb0 - wan port
igb1 - lan port >netgear GS108T
igb2 - TPLink AP
igb3 - TPLink AP
LAN - BRIDGED 0 (igb0, igb1, igb2, igb3)
Given the NIC hardware capabilities, that nearly all traffic is outbound and the only REAL ‘switchy’ traffic would be on the managed switch; I figured I would be good to go.
But no… as soon as you mention bridging, every pfsense user in the world simultaneously self immolates and screams ‘its not a switch!!!’ So why does netgate make the sg2100??? 
Anyway, the above config actually works extremely well (no cpu hit, no performance hit) BUT it all fell to pieces when I tried to introduce VLANs. My digging so far seems to point to a freeBSD issue ( bridged parent interfaces can’t pass tagged and untagged traffic) that has recently been patched up but probably hasn’t been pushed to pfsense yet.
You have videos on bridging (Suricata I think?) on this hardware, I was wondering if you had done any with VLANS
I believe the correct procedure to be:
with these tuneables, rules on the individual interface/vlans should be ignored/not needed. This should be inline with Netgates ‘assigned bridge’ vs ‘simple bridge’ wireless example - pfSense wifi AP video
Might work, but as I said I don’t recommend using the 4 ports as a switch, if you need a switch get a switch. The SG-2100 has a special type of chipset designed to act as a swtich with the drivers written for that function.
Because on devices like the SG2100, SG3100, etc, there actually is a real switch built in. It isn’t just 4 ethernet ports available to the CPU like if you used a computer with a NIC card. Traffic between LAN ports on the SG2100 doesn’t go through the CPU, but with a NIC card with 4 ports that are bridged it will. The switch built in to devices like the SG2100 has an extra internal port that is connected to the CPU. The software running on the router is really only dealing with that single internal port, and splitting traffic out to the front panel ports has to be done via VLANs - even if you don’t intend to use VLANs, they will be set up automatically to handle the traffic to/from and within the switch chip.
This is actually the difference between a “switch” and a “bridge”. Switch is the term used when the traffic is handled by an ASIC, Bridge is the term used when the traffic is handled by software.
I think I’d argue that bridge just connects 2 or more ports together and a switch makes decisions where the data should go based on the L2 MAC address. Not really to do with hardware vs software.
However your point still stands, an actual switch is going to do a better job and use less processor cycles than “bridging” 4 separate ports and having the main processor do the switching.
A bridge makes decisions about how to forward traffic via Layer 2 MACs as well. Switches didn’t initially exist, because modern ethernet came around comparatively late. Bridges were invented as an alternative to hubs, which just copy every packet to every other port (early hubs didn’t even have electronics in them to receive the packet and then retransmit it, they just connected all the ports together and amplified the signal). Bridges improved this by reading the MAC and only forwarding the packet to the correct interface. They were expensive and slow because they ran on commodity hardware and operating systems - just like most routers at the time. To make this faster and cheaper, ASICs for this task were invented. These new devices which used an ASIC instead of a CPU and operating system were then called Switches.
Resources: https://geek-university.com/ccna/differences-between-a-switch-and-a-bridge/
https://web.archive.org/web/20100105152318/http://www.networkcomputing.com/1119/1119f1products_5.html
Few thoughts here…
and compare to this Cisco Nexus diagram:
First I see the same basic setup: intel cpu (albeit weak sauce embedded), 16gb ram, pci bus, an ASIC and PHYs. The Cisco is a big dog switch bestowed with an ASIC full of machine learning algo’s and other PFM sorcery… Not many budget/home labs are moving enough data to even bother with something on this level let alone afford the current $5k ebay price.
So take a look at this more basic switch block diagram. Hmm… That looks ALOT like the Intel NIC
Are these NIC’s switches? They are not simple PHY / LAN bridge ports, they have vlan tagging/trunking/ have buffers/hardware flow control, the all important ASIC etc etc… So yes? Maybe we should stop calling them NICs and start calling them SICs. Are they going to be as fast or feature rich as a stand-alone dedicated device? Probably not a good one, no. However, I would expect these cards to handle intra-port switching tasks on par with any stand alone device given the same link speed and number of ports.
Scaling wise, I imagine adding a second card and expecting any sort of performance through the PCIx bus is silly. Maybe that is what people have in their heads when asked about this type of application.
TL/DR: Is the intra port switching speed adequate for budget deployments/home lab environments? If yes, maybe we should all evaluate the use case before screaming ‘get a switch!’.
Summarizing this, because I know I’m getting repetitive: there are many functions that a managed switch can do, such as VLANs and traffic shaping, which some NICs for a computer also have. But those NICs (and ASICs for them) do not handle forwarding of packets at layer 2 directly between the interfaces, which is why they are not a Switch or “Switching ASIC”.
When you bridge ports in order to have packets forwarded between them, you are forcing the packets to be handled by the CPU/software. The CPU load to handle one packet being forwarded on the bridge is nearly as much as the CPU load to handle a same-sized packet being routed with basic firewall rules (for systems using FreeBSD or Linux kernel bridges). Bridging is easier, but it is about the same order of magnitude. Therefore whether bridging provides adequate speed depends on the hardware the user has, and also how much routing and other functions they have running.
When providing free help on the internet, fairly quickly one decides to take the shortest path. Working with every new user to get them to understand the limits of bridging is tiresome. Oftentimes, they are only coming to a forum/reddit/discord after having set up a bridge and are complaining about speed - that they can’t get gigabit LAN transfers. So in general when someone asks about it, I try to point them towards switching. There is a time and a place for bridging - I have used it in many cases both personally and professionally. And just one example - every wireless AP uses bridging to connect the interface(s) for their wireless radios to the ethernet port. I haven’t seen any device which uses an ASIC to directly do switching between the radio and the ethernet ports. Since there are APs that can exceed 1Gb/s, clearly bridging on its own is not slow - it is just more dependent on the performance of the CPU. In a closely controlled environment like an AP, where bridging is nearly all the CPU will do, the performance can be good and consistent. On a router where both the amount of traffic for bridging and for routing/firewall are variable, and there are other processes like DHCP and DNS, it is hard to be sure what performance you will have.
Every days a school day.
Until today I thought that a bridge was a two port hub and that switches were hubs with L2 “intelligence” when in fact, a Hub is a two port Switch…
Sorry for questioning your statement, happy to be corrected!
You’re on the right track, but still wrong. A hub is not a switch - a hub is just a hub. Hubs don’t look at MAC addresses at all, neither to learn which MACs are on which port nor to decide where to forward a packet to. Every packet that comes in, is copied to every other port.
A bridge/switch will watch the source MACs of the incoming packets to learn which MACs are on which interfaces. It then uses that “Forwarding Database” table to know where to forward packets based on the destination MAC. If the destination MAC isn’t in the table, then it will flood it out every interface. Hopefully this doesn’t happen often. Anything sent to the broadcast MAC (FF:FF:FF:FF:FF:FF) is sent to all interfaces.
Gotta say I’m being schooled 
If the costs are marginal I’d go for the Quad port each time.
Typo, I wrote hub but meant bridge!
I was looking at the SG-2100, I wondered if I buy an off the shelf SATA M.2 with larger than the weirdly sized 32gb m.2 provided by NetGate would it be compatible? Or did netgate do something funky to this router where it will reject custom hardware?
Maybe @LTS_Tom would know?
https://www.newegg.com/western-digital-blue-250gb/p/N82E16820250090
I have never replaced or upgraded the drive in a Netgate device.
You may have already checked This. It’s for an SG-3100 but the author mentions something about compatibility.
Perfect, thanks a lot! I hadn’t found that one.
Yes I’m going to necro my own thread…
I enjoy spirited debates but the trick is to recognize when the other parties are no longer having fun; so I dropped this. I recently borked my pf box and the down time spawned some new hardware purchases and this topic popped back into my head.
From the Intel VMDq Overview:
A key part of the I/O container network subsystem is the virtual switch or virtual bridge component. The terms virtual switch or virtual bridge (here referenced as virtual switch) are often used interchangeably to describe a layer-2 network device that is used to join LAN segments.
The virtual switch interconnects the virtual and physical LAN segments at the network interface layer (I.E. Layer 2) and forwards frames between them. The virtual switch acts as a MAC relay and is independent of any upper protocols.
The virtual switch is “transparent” to the Internet Protocol (IP) layer. For example, when a host sends an Ethernet frame to another host on a network connected by a virtual switch, the host sends the frame directly to the targeted host and the frame “crosses” the virtual switch without the sending or receiving hosts being aware of the virtual switch.
…
One of the main tasks of a virtual switch is to sort network traffic based on filters and to forward the traffic to the appropriate destination. The virtual switch is transparent to its communication partners. None of the host systems are aware that the virtual switch is present in the network path. To achieve transparency, the physical NICs used by virtual switches are configured to enable promiscuous mode. This allows the switch’s physical interfaces to receive incoming packets destined to any MAC address. The virtual switch inspects each packet’s destination address and, using layer 2 filters, decides whether a packet should be forward to another specific virtual switch interface or dropped.
Interestingly though, it seems older 2 port versions of the card may be better suited and more secure as the Intel 82580EB Data Sheet seems to indicate these i340-T4 cards only supported VMDq whereas 82576 based cards (dual ports like E1G42ET ?) supported VMDq2 which seems to be at the core of more current convergence cards er what ever I just glanced at on Intel’s site.
Thoughts?
BTW I did get my bridged setup working and working well however I was messing with vlans/rules one day and DHCP shat the bed. It seems DHCP on bridges requires some hidden secrete sauce config I’m too n00b to sort out so I’m re-wiring the house (one of the main reasons for the bridging was physical layout/cabling issues) and going with a more traditional install.
VMDq / VMDq2 aren’t enabled by default, the OS has to instruct the NIC to use those features and specifically what it expects it to do on its behalf. It is like other NIC features such as TCP checksum offload, packet fragmentation/reassembly offload, etc. The OS would have to know how to use this. This is also going to be much more basic than all the features possible in a bridge. This is something you see a lot from Mikrotik, where they either allow the admin to choose between hardware accelerated functions or software accelerated, or they build in logic such that they use hardware acceleration/offload unless you enable an option that can only be done with software. Another example would be Ubiquiti EdgeRouter/USG, where they have the option of doing hardware-accelerated routing or purely software routing. You don’t see this in PFSense, if they’re going to use hardware features, like their routers with builtin switch chips, then that will be the only way to set up those ports. They do have some ability to use more generic NIC offloading, but only functions that exist on nearly every NIC from every vendor.
