Inbound NAT (port forwarding) with multiple VLANs (gateway problem)

Hi there. I am trying to forward two ports from my firewall’s public-facing interface to a server in my local network and ran into a problem. I have two LANs, and Both services run on the same server, but are bound to either of the LANs (cf. graphic).

In pfSense, I have setup the two respective port forwarding rules which I confirmed are working. However, I could only access the service from the internet. Initially, I had setup only one gateway on the server (on the network), like so:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface         UG    0      0        0 ens160   U     0      0        0 ens160   U     0      0        0 ens192

During troubleshooting I added a gateway on the other interface as well, which didn’t change anything on its own. It was only after making the default gateway (see below) that I could access the service listening on from the internet. However, now the other service wasn’t working anymore.

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface         UG    10     0        0 ens192         UG    20     0        0 ens160   U     0      0        0 ens160   U     0      0        0 ens192

So this seems to be a problem with the network configuration of the server, not pfSense. Any help on how to get both services exposed at the same time is greatly appreciated!

The server is an Ubuntu 18.04.2 LTS running on ESXi. pfSense is also running as a VM on the same host. Network config on the Ubuntu is done with netplan.

I am a bit confused, are you trying to forward that same port to both servers?

1 Like

It’s a little bit confused, but you don’t need 2 default gtw because you have ip’s of 2 networks in the pfsense.

review your nat rules.

the initial problem may be that you don’t answer in the same interface that start the request.

No, sorry, I forgot to mention:

Port 80 on the public IP is forwarded to
Port 8080 on the public IP is forwarded to

As I said, I can confidently rule out (no pun intended) NAT misconfiguration. Both rules work individually, depending on the configuration of the server. Why would the NAT rule be influenced by the network config of another device? Except if I’m missing something here about how NAT operates.

About not needing two default gateways, I agree. I only did this during troubleshooting.

Alttough I don’t fully understand what you mean, i feel you’re onto something here. My guess is that I have to configure the server to send the replies out a specific interface. In other words, I have to add some routes. But I don’t know which.

I am now absolutely positive it is a problem with the server configuration, not with pfSense.

When I have a machine in any one of the two given networks, I can’t ping or connect to services listening on the other interface of the server, respectively. Only the ones on the interface in the same network. This is not a firewall issue either, I have verified that. The server is a fresh install of Ubuntu with docker installed.

I think if I figure out how to solve this problem, the NAT will work.

Hi. I’m also a bit confused :stuck_out_tongue: You state in the topic that this is two vlans. So my question is if you have set up the vlans correctly and you have a Vlan aware switch? And are there 2 physical cables plugged in the switch or are the vlans stripped on the same interface? (pfsense gw vlan “xxx”) <- (pfsense gw vlan “xxx”) <-

The port your pfsense is plugged in to need to be a trunk port so you can access both vlans.

Or did I miss understand your setup?

Indeed. Your multi-homed server probably sends it’s replies on the other interface than it receives it’s requests. Normally that isn’t a big problem, but your firewall doesn’t like this.

See Ensure symmetric routing on a server with multiple default gateways for more information on the topic. They also present a possible solution.

Hey JobH, sorry for the late response. Thanks for your answer! I’ve checked it out and it looks promising, however my Ubuntu server uses netplan for interface configuration and it seems to be conflicting with the solution solution described. I’ll have to put some more thought into this.

I did not try this myself, but the netplan source routing example seems to contain the information necessary: