Weird SSH behaviour in my pfSense firewall

Hi folks,

Im having a issue thats been driving me nuts for months. Bear with me here.

I have a Raspberry Pi 4 server running Ubuntu 22.04 and LXD on my 10.33.2.0/24 network.
The ufw firewall on the Pi 4 is INACTIVE.

I have a laptop, on the same network, in the 10.33.2.0/24 range.
The ufw firewall on my laptop is INACTIVE.

When I put my laptop on a different network, (for example 10.33.5.0/24 range) I can ssh into my Pi 4 server without issue, but when Im on the SAME network as the Pi 4 server, I cannot access it.

I ran Wireshark on the ethernet port and I only get an SYN, no ACK back.

In my pfSense router, that handles all routing on my network, I’m seeing the following entries in the firewall log:

Yet, when I go to my LAN firewall rules, I have the following wide open scenario:

Can anyone point me in any direction here so can I can delay the greying process?
Thanks for your time.

On PFSense can you ping the PI ip address

Want is the LAN address range ?. is this the network the Pi4 is on.

I can ping the Pi address from pfSense.
I can ping the Pi address from my laptop.

Yes, the Pi and the laptop are on the same network, which is the LAN.

On PFsense, under Advanced → Admin Access, have you changed the port for ssh for PFSense

Do you have a floating rules ?

If you’re having issues communicating on the same LAN the router/firewall has nothing to do with it. It operates at layer 3 and above so only gets involved in communication between different networks. After all, you can have 2 computers communicate just fine with each other using a single cable or two cables and a switch, no routers/firewalls involved. I would double check your ip and network mask configuration on the two devices that can’t communicate. I’ve seen it before where a subnet mask has been mistyped and that prevents the devices talking because they are quite rightly on separate networks, even though they are on the same ‘physical’ network.

Yes, I have changed the port for ssh for pfSense.

There are some floating rules, mainly for pfBlockerNG. I have disabled these rules and still cannot access my Pi.

Yes, what you say makes perfect sense.

This is the netplan config on my Pi 4. It seems to work fine, in that I can access this Pi from other networks, just not the same network that it exists on.

network:
    version: 2
    ethernets:
        eth0:
            dhcp4: false
            dhcp6: false

    bridges:
      br0:
        interfaces: [eth0]
        addresses: [10.33.2.10/24]
#        gateway4: 10.33.2.1
        routes:
        - to: default
          via: 10.33.2.1
          table: 800
        routing-policy:
        - from: 10.33.2.10/24
          table: 800
          priority: 300
        mtu: 1500
        nameservers: 
          addresses: [10.33.2.1, 1.1.1.1]
          search: [local.lan]
        parameters:
          stp: false
          forward-delay: 4
        dhcp4: false
        dhcp6: false
 

This is the output from ip route

user@pisrv:~$ ip ro
default via 10.33.2.1 dev br0 proto static 
10.33.2.0/24 dev br0 proto kernel scope link src 10.33.2.10 
10.146.74.0/24 dev lxdbr0 proto kernel scope link src 10.146.74.1 linkdown 

I really don’t understand why I’m getting the following two entries in my pfSense firewall logs when I try to SSH from my latptop which is in the same subnet as the Pi 4.

It is odd, as the firewall shouldn’t see any of the traffic from your laptop going to your server, there is no need for it to go via the firewall. What is the ip configuration on the laptop? As it would appear that it thinks it needs to go via the firewall even for devices on the local network.

The laptop is running Ubuntu Mate 22.04, but it doesn’t matter what machine I sit at, Im unable to connect into the Pi 4.

image
image

This is cracking me up a bit. I can ssh into other hosts on the same LAN network from my laptop just fine. Its just that one Pi 4 LXD server.

Every time I try to log into that Pi 4 server, those entries appear on the firewall for that one host. No entries appear for other hosts that I try to connect to. Thanks for listening to my idle ramblings. Your comments have encouraged me to search in different areas.

I came across this post but not sure how to move forward with it…
Something to do with the firewall retaining old states TCP [SYN:ACK] stuff. I did wireshark the connection from my laptop, the laptop sends out a single SYN, but gets no ACK back.

I have flushed the states in the firewall,
I have restarted the Pi 4 server,
I have restarted my laptop,
I have run out of ideas.

I may be a bit out of my depth here.

As I said before, the firewall should not take part in the conversation unless you are going from one network to the other. So I would suggest you take it out of the problem by removing it. Can you get the laptop to talk to the server when you just have the two connected via a switch? If you can’t the firewall isn’t the issue.

Fair enough. Testing setup.

Pi 4 ------- switch ------ laptop.

So I set up a switch with my Pi4 (which is running LXD with a single container).
I can ping the container from the laptop but not the host (the Pi4). I also cannot ssh into the host.

I edited out some lines from my 50-cloud-init.yaml file on the Pi4 and I can now ssh into it.
I can’t say I fully understand why but there you go.

    version: 2
    ethernets:
      eth0:
        dhcp4: false
        dhcp6: false

    bridges:
      br0:
        interfaces: [eth0]
        addresses: [10.33.2.10/24]
          #        gateway4: 10.33.2.1
        routes:
        - to: default
          via: 10.33.2.1
            #          table: 800
            #        routing-policy:
            #        - from: 10.33.2.10/24
            #          table: 800
            #        priority: 300
        mtu: 1500
        nameservers: 
          addresses: [10.33.2.1, 1.1.1.1]
          search: [local.lan]
        parameters:
          stp: false
          forward-delay: 4
        dhcp4: false
        dhcp6: false

It also now appears on the list of found devices when I do an nmap -sP 10.33.2.0/24

Thanks for your interatctions and encouraging me to dig around.

I can’t say I know LXD at all, but I’m glad you managed to figure out a fix. I suspect it was routing all traffic out via the firewall, so if you came in from there it would work, but if you came in from the local lan the return traffic was going to the firewall and not the original device. Just a guess on my part.

2 Likes