Pfsense in Lab, no internet?

I’m sure I am just missing a really simple setting but I can’t seem to figure what is creating this wierd issue.

So I have a dedicated pfSense running at the head of my network which runs great but I’ve been trying to setup pfsense in a xcp-ng vm for testing and other things.

I’ve installed pfSense running with fresh default settings. Using the following networks

  • WAN - (DHCP, from head end using a /24 network)
  • LAN - (XCP-NG private network interface, Running DHCP for clients)

When I put my test desktop on the LAN interface it grabs a ip and can talk to the vm pfsense but can not connect to the outside world expect for ping/dig. However the lab pfsense has full internet and can ssh/ping/dig out.


PING ( 56 data bytes
64 bytes from icmp_seq=0 ttl=54 time=13.170 ms
64 bytes from icmp_seq=1 ttl=54 time=14.113 ms
64 bytes from icmp_seq=2 ttl=54 time=13.538 ms

— ping statistics —
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 13.170/13.607/14.113/0.388 ms

PING ( 56 data bytes
64 bytes from icmp_seq=0 ttl=55 time=9.866 ms
64 bytes from icmp_seq=1 ttl=55 time=7.468 ms
64 bytes from icmp_seq=2 ttl=55 time=6.995 ms ping statistics —
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 6.995/8.110/9.866/1.257 ms

[2.4.5-RELEASE][admin@pflab.localdomain]/root: ssh r***.ca
Permission denied (publickey,gssapi-keyex,gssapi-with-mic).

Desktop behind lab.pfSense:

└──╼ $dig
; <<>> DiG 9.11.5-P4-5.1ubuntu2-Ubuntu <<>>
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 52867
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

; EDNS: version: 0, flags:; udp: 65494
; IN A


;; Query time: 142 msec
;; WHEN: Tue Mar 31 15:30:55 EDT 2020
;; MSG SIZE rcvd: 54

└──╼ $ping
PING ( 56(84) bytes of data.
64 bytes from ( icmp_seq=1 ttl=54 time=16.4 ms
64 bytes from ( icmp_seq=2 ttl=54 time=10.5 ms
64 bytes from ( icmp_seq=3 ttl=54 time=9.83 ms
64 bytes from ( icmp_seq=4 ttl=54 time=10.1 ms
64 bytes from ( icmp_seq=5 ttl=54 time=15.8 ms
64 bytes from ( icmp_seq=6 ttl=54 time=16.3 ms
^C ping statistics —
6 packets transmitted, 6 received, 0% packet loss, time 5007ms
rtt min/avg/max/mdev = 9.834/13.140/16.399/3.035 ms

└──╼ $ssh r***.ca

clog -f /var/log/filter.log while trying to make outbound connections

[2.4.5-RELEASE][admin@pflab.localdomain]/root: clog -f /var/log/filter.log
Mar 31 19:17:37 pflab filterlog: 7,1000000105,xn1,match,block,in,6,0x00,0x00000,1,Options,0,56,fe80::981f:fc58:f7ba:7da1,ff02::16,HBH,RTALERT,0x0000,PADN,

So in TL;DR;

  • pflab.localdomain can access the world just fine
  • anything behind pflab.localdomain can ping/dns/dig out but no tcp/udp packets but nothing in the firewall?

Is this a double nat issue?

I’ve literally been doing the same thing this evening with pfSense on a vm in xcp-ng. I even had similar issues. I had no internet access from my lab pc, couldn’t ping or do anything, but all tests worked fine from the pfSense vm. My issue turned out to be the rules on my physical pfSense box blocking dns traffic (I have it locked down to only allow pfSense to act as a dns server). I had to set my vm pfSense to forward to my physical box. I couldn’t see the issue in the logs but if I created an allow all rule things worked.

Put a temporary allow all rule on your physical box to rule that out, and if it works try narrowing it down one step at a time.

Thanks for the reply.

I too have a dns rule on the main pfSense, however I have it disabled for testing and dns is working on the lab pfSense box.

I’ve been having a play with my setup again and I can confirm that I have full Internet access from my lab even with the double NAT on two pfSense firewalls. I know that doesn’t really help you much, but at least you know it should work. I have however had random periods of the network not working, can’t ping etc. then it just starts working again, not sure if that is pfSense or XCP-NG

That’s good to hear its working for you :smiley: No difference from my end.

@LTS_Tom Would it be possible to get a backup xml of your pfsense that you used for the two public wan ip’s that you used in your video to compare to what I have?

I already purged those, but if you are using pfsense in a double NAT lab setup you may have to turn off the options “Block Private Netowrks” option on the WAN of the lab system. If you don’t it will block traffic from IP addresses that are reserved for private networks per RFC 1918.

1 Like

Already off, and I would assume if it wasn’t it’d show up in the /var/log/filter.log?

I’m kinda lost to what is causing this issue.

Sloved! Turned out I needed Disable hardware checksum offload to be checked.

1 Like

Well done! I was actually just coming on here to post I’ve had to do exactly that on my test system to get it to be more stable. Strange as I’m using Intel NICs which are normal fine, but it’s only a test system so I don’t mind. So it looks like our problems were the same thing after all!