Thank you. Based on what you said I went back and troubleshooted a little further, and it turns out iptables was misconfigured.
For others who might have the same issue, here’s how it was resolved:
My setup looks as follows:
eth0: WAN interface
eth1: LAN interface
pia: PIA VPN over WireGuard
I set up the VM such that it would route packets from LAN (eth1) to pia, and iptables was initially set up as follows:
iptables -t nat -A POSTROUTING -o pia -j MASQUERADE
iptables -A FORWARD -i eth1 -o pia -j ACCEPT
iptables -A FORWARD -i pia -o eth1 -m state --state RELATED,ESTABLISHED -j ACCEPT
However, I was having the issues I described in my initial post. The symptoms were weird. From a client machine on the same IP range of eth1 (purely switch traffic), certain websites worked ok (but relatively slow to handshake), while others (like Reddit, Bank of America, etc) did not work at all. It turns out iptables somehow was misbehaving when transmitting packets from eth1 to pia and back (apparently the MTU settings from pia were not being “observed” by iptables when routing packets back and forth).
What was missing was the following additional configuration:
iptables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN -o pia -m tcpmss --mss 1360:1536 -j TCPMSS --set-mss 1360
Below is the explanation I got from X’s Grok about that command:
This iptables command is used to manipulate the TCP Maximum Segment Size (MSS) for packets going through a network interface named pia. Here’s a breakdown of what each part of the command does:
-t mangle: Specifies the table to use. The mangle table is used for packet alteration.
-A FORWARD: Appends a rule to the FORWARD chain. The FORWARD chain is used for packets that are being routed through the current host to another destination.
-p tcp: Specifies that this rule applies to TCP protocol packets.
--tcp-flags SYN,RST SYN: This matches packets where the TCP flags SYN and RST are set, but only checks for SYN packets. This is typically used to target connection initiation packets (SYN packets).
-o pia: Specifies that this rule applies to packets that are going out through the interface named pia.
-m tcpmss --mss 1360:1536: This uses the tcpmss match to target packets where the MSS value is between 1360 and 1536 bytes. MSS is the maximum size of the payload of a TCP segment, excluding the TCP header.
-j TCPMSS --set-mss 1360: If all the above conditions are met, this jumps to the TCPMSS target which adjusts the MSS of the TCP SYN packets to 1360 bytes.
Purpose:
This rule is likely used for:
Path MTU Discovery: By setting the MSS, you're effectively controlling the maximum size of the TCP payload to ensure that packets do not exceed the Path Maximum Transmission Unit (MTU), which can help avoid fragmentation or packet drops due to MTU limitations along the network path.
VPN or Tunneling: Given the interface name pia (which might stand for Private Internet Access, a VPN service), this could be part of configuring the network to work optimally with a VPN. VPNs often require adjustments like this because they encapsulate traffic, which can reduce the effective MTU of the path.
Avoiding Fragmentation: By reducing the MSS, you ensure that the encapsulated (or tunneled) packets plus the VPN overhead still fit within the MTU of the physical network interfaces, reducing the need for fragmentation which can degrade performance or cause issues if fragmentation is not handled well by all devices in the path.
This setting helps in optimizing TCP traffic over networks where the standard MSS might lead to issues due to lower MTU, especially common in scenarios involving VPNs or other forms of tunneling where additional headers reduce the available space for payload.
After putting in this additional settings, all https sites work, and initial handshake for all websites feel much faster.
Hope this helps somebody in the future trying to do the same!
Cheers!
