Site-Site WireGuard pfSense-Mikrotik vLAN Server access Issue

Hi there all,
I wonder if you would perhaps assist me. I need some insite.

I am running pfSense in my own enviroment. The remote site is running a Mikrotik router.

  1. WireGuard setup and established between pfSense and Mikrotik
  2. WireGuard IP for pfSense 172.16.172.2
  3. pfSense static addressed
  4. vLAN30 IP range 192.168.30.0/24
  5. WireGuard IP for Mikrotik 172.16.172.3
  6. Mikrotik CloudDNS no static IP
  7. native vLAN on the LAN range 192.168.0.0/24
  8. ICMP both ways on WireGuard IP range
  9. I have a server that I need to access the vLAN30 range behind pfSense.
  10. I have the ability to access the server on my pfSense native vLAN1 throught my entire network behind pfSense BUT I am unable to access the 192.168.30.0/24 network from the remote site.

Firewall rules on pfSense:
WireGuard Net
Block/Allow Prorocol Source Port Destination Port Gateway
Block IPv4* * * LAN net * *
Block IPv4* * * Home net * *
Block IPv4* * * Server net * *
Allow IPv4* * * * * *

vLAN30 Net
Block/Allow Prorocol Source Port Destination Port Gateway
Block IPv4* * * LAN net * *
Block IPv4* * * Home net * *
Block IPv4* * * Server net * *
Allow IPv4* * * * * *

I have not worked with Mirotik but check that you have the proper routing table on each device to allow getting to the 192.168.30.0/24 networks.

Hey ShaneSerrao,

You didn’t mention what the remote site’s subnets are (to complete the “network diagram”), and whether you’ve got ICMP across the subnets as well, not just along the WireGuard tunnel net.

  • Like Tom said, confirm routing tables from both ends.
  • Confirm no subnets clashes across the sites.
  • Confirm ICMP, between the tunnel end-points (which you’ve done) as well as over the tunnel from other hosts behind the WireGuard end-points.
  • Temporary disable your rules, or add an allow all to the top while troubleshooting.
  • You didn’t say what sort of server you’re trying to access, but be aware that Windows for example will block traffic for non-local subnets depending on whether it’s on a Domain, Public or Home net.

Hi @LTS_Tom and @stildalf

Thank you .

So the mistake was on the routing table.

@stildalf, to respond to your question.
I want to access Zentyal DC based on Ubuntu 20.04.4.

  • a thing that I still need to figure out as I have never done this before is to allow access to the DC on a outside net via WG.

  • Very weird thing that I am experiencing. I can ping the DC IP address but I can’t ping the .local domain name nor the host system where the VM is should live on a temp basis. (ICMP to 192.168.30.253 but no ICMP to 192.168.30.242 BUT there is ICMP traffic to the remote site from any terminal)

Didn’t have any time to dig into it as of yet.

Glad you got your routing sorted.

Some comments re your last post though.

  • I don’t want to discourage you from using Zentyal in the manner you describe, just to be aware of some of the hurdles you’ll encounter:
    • You can certainly run AD DC (Zentyal or Windows) behind a VPN but just be aware that, specifically domain members, are going to need the DC as their primary (preferably only) DNS server.
    • Even then name resolution within your domain will still be sub-optimal if the DHCP server is separated from the DNS server. Typically DHCP & DNS run tightly integrated, especially in an AD environment, where DHCP informs DNS re your dynamic hosts.
    • Depending on the amount of domain members and latency to the remote site, you could even consider running an Additional Domain Controller with Zentyal remote side. Everything AD wise, except GPOs, will sync just fine with Zentyal. GPOs can be synced easily enough with rsync or something similar.
  • I would avoid using that .local domain
    • Yes, even MS historically used it by default - but there’s long since been other services that make use of and muddied the waters around .local - mostly mDNS and it’s ilk.
    • Ideally use a domain name you actually own or pick another Top-Level Domain that’s not in use - but therein lies the rub, you never know when said TLD might become active.
    • I still begrudgingly have some .lan’s lying around, and they seem safe for the moment.
    • Other names to avoid: .net; home; .dev; anything .inuse…

@stildalf, Thank you for the insight.

  • I am aware that the AD should be the primary and preferred only the AD as a DNS. I really wish that was possible. We have such an unstable power grid at this very moment, on the bad days we are experiencing rolling blackouts of up to 8 hours per day split into 2 or 4 separate schedules hence I need to use a secondary for the in-case when the blackouts hit. The remote site has to continue with operations as and when it can.

*Due to that, I have no choice but to run DNS and DHCP apart from one another. I will need to figure this out. (If you might have some pointer for me, it would be appreciated).

  • On the upside of things. Fortunately both sites has got a decent fibre connection, the RTT from Main to Remote on GW-GW is in the region of 10MS which makes life somewhat better. Another fortunate only 9 Domain members on the remote site.

  • Regarding the TLD, I will implement a long term solution. So it would be wise to perhaps use a TLD that isn’t registered but truly available… would a .xyz domain work? I know that its chap as chips and not too much preferred in the market.

Do you have any experience with Zentyal?
I want to find out, if the Mail server is any good? how does it compare to MS Exchange?

Very much sounds like we’re in the same South African Load Shedding Boat - I’m right now sitting here listening to the (not so) soothing sounds of the generator running outside.

  • .xyz is an active TLD since 2014, so I wouldn’t use that either, unless you register some subdomain there of course. (Sorry, I see now that’s perhaps what you meant, then yes, get a domain that suits your organisation, the costs mostly don’t feature here.)

  • I do have some experience running Zentyal for a number clients, but mostly in an AD role - replacing what was previously self-rolled Samba or NT4 domains from yesteryear.

  • I don’t personally use Zentyal’s mail or exchange features - it just never felt like they ever managed to settle on something for a reasonable amount of time. (To be fair, that is a tall order for anyone looking for an exchange replacement, and by no means the fault of Zentyal’s devs)

    I’ve run through a decent gamut of mail servers myself - from Exchange to MDaemon on Windows (what feels like a lifetime ago), to Scalix then Zimbra (perhaps half a lifetime ago) then Zarafa and currently Kopano on various Linux flavours. For the moment Kopano still serves my clients well - but I’m never likely to stop assessing/looking for viable alternatives - especially in the self-hosted space.

@stildalf
Well nice to meet a fellow south African here.

Would you perhaps have some insight on the following issue:

  • .co.za TLD Domain Name
  • I am able to ICMP the .co.za domain with no problem from anywhere in the network

Now here comes the thing:

  • ICMP to 192.168.30.253 /.co.za domain name from
  • 192.168.30.0/24 ‘server range’
  • 192.168.0.0/24 remote Native vLAN
  • 192.168.1.0/24 local native vLAN
  • BUT no ICMP to the Zentyal VM host 192.168.30.242 or 192.168.30.252 from anywhere but the vLAN30 net furthermore:
  • From the vLAN30 net I am able to join the .co.za DC
  • Created a Win10 VM on the native vLAN behind pfSense where I am fully able to ping 192.168.30.253/.co.za domain name
  • But I am not able to join the .co.za domain at all. (note that the .co.za domain isn’t a registered domain so it is available for use/registration)
  • Firewall rules for testing purposes:
    1 WireGuard Net Allow IPv4* * * * * *
    2 vLAN30 Net Allow IPv4* * * * * *
    3 native vLAN Allow IPv4* * * * * *
  • Zentyal DC filtering rules for internal networks
    1 accept |any source |any Destination |any Service

The error received on the Win10 VM on the local native vLAN range

The error was: “DNS name does not exist.”
(error code 0x0000232B RCODE_NAME_ERROR)

The query was for the SRV record for _ldap._tcp.dc._msdcs.****.co.za

Common causes of this error include the following:

The DNS SRV records required to locate an AD DC for the domain are not registered in DNS. These records are registered with a DNS server automatically when an AD DC is added to a domain. They are updated by the AD DC at set intervals. This computer is configured to use DNS servers with the following IP addresses:

192.168.30.253

One or more of the following zones do not include delegation to its child zone:

****.co.za
co.za
za
. (the root zone)

You’re invariably gonna have to employ some form of Split DNS in this case, but I’ll describe a scenario for you in another post with some more details to look out for. Maybe something hits home.

Just to clarify: .za is the TLD in this case, more specifically a ccTLD - country code top-level domain. I would avoid using any sub-domain of a valid TLD, unless you actually plan to register said domain (in that case, grab it quickly).

We use Split DNS whenever we have to present different records internally vs what’s available publicly.

Thank you,

Yes the Sub-domain is available and I plan to register the sub-domain for this use only.

Ok, so I’ve made quite a few assumptions here, but let’s say you’ve registered (or plan to) demo.co.za, here’s a possible scenario for you. Sorry if some of this seems irrelevant, or sparse in places.

SiteA

  • 192.168.1.0/24 - Native VLAN - LAN1
    • .1 - pfSense, GW, DHCP Server
  • 192.168.30.0/24 - Server VLAN - VLAN30
    • .1 - pfSense, GW, DHCP Server
    • .242 - VM Host
    • .253 - Zentyal VM, AD DC

SiteB

  • 192.168.0.0/24 - Native VLAN
    • 192.168.0.1 - mikrotik, GW, DHCP Server

SiteA pfSense config

  • General Setup
    • hostname: pfSense
    • domain: lan1.demo.co.za
  • DNS Resolver
    • Domain Overrides:
      • Domain: ad.demo.co.za > 192.168.30.253
    • Access Lists:
      • DEMO Nets - Allow: 192.168.1.0/24; 192.168.30.0/24; 192.168.0.0/24;
  • NTP Server
    • Either don’t select any interfaces, or bind to all your internal IPs
    • Either use the default Time Server pools or for ZA use [0-3].za.pool.ntp.org
  • DHCP Server on LAN1
    • Specify your range, the rest can mostly be left empty/defaults except perhaps:
    • NTP Server 1: 192.168.1.1
    • Domain Search List: lan1.demo.co.za; ad.demo.co.za
  • DHCP Server on VLAN30
    • Specify your range
    • NTP Server 1: 192.168.30.1
    • Domain Name: vlan30.demo.co.za
    • Domain Search List: ad.demo.co.za; lan1.demo.co.za

SiteA Zentyal config

  • Domain
    • Server Role: Domain Conroller
    • Realm: ad.demo.co.za
    • NetBIOS domain name: DEMO
    • NetBIOS computer name: dc01
  • DNS Forwarders
    • 192.168.30.1

SiteB mikrotik config

  • DHCP Server
    • Specify your range
    • Domain Name: ad.demo.co.za
    • NTP Servers: 192.168.30.1
    • DNS Server 1: 192.168.30.253
    • DNS Server 2: 192.168.0.1
    • Domain Search List: ad.demo.co.za; lan1.demo.co.za

DNS and Synced Clocks are very important here. You’re going to experience domain resolution issues at SiteB whenever the VPN goes down - any DNS caching on those domain members and the mikrotik router is gonna prolong recovery post VPN return.

You’ll need to confirm name resolution for *.ad.demo.co.za is handled by Zentyal, hence the domain override. Essentially, if you can successfully resolve the ad realm from a subnet, you should be able to join the domain from there. Test with nslookup/dig to dc01.ad.demo.co.za and _ldap._tcp.dc._msdcs.ad.demo.co.za to confirm.

I’ve got limited experience with Mikrotik and RouterOS, but found the DNS config options rather sparse - obviously spoiled by Unbound & pfSense.

Another suggestion I’d like to throw in the mix, is to move away from the typical default subnets you find pre-configured on most devices as soon as possible. 192.168.0.0/24, 192.168.1.0/24, 10.0.0.0/24 (or :face_exhaling: /16). If you ever need to add another site, or provide roadwarrior access to users behind their default routers, you’re gonna have some fun juggling your routing tables.

@stildalf .

So, I want to really thank you for the retailed response.
I was able to join a VM from 192.168.1.0/24 after I have followed your instructions.
I will have a look tomorrow @ SiteB if I am able to do the same.

I had 3 things not configured.

  • NTP Server - The pfSense Firewall wasnt the prime on all of the interfaces and also not on the Mikrotik.

  • DHCP Server on LAN and VLAN30 -

    • Domain Name: vlan30.demo.co.za
    • Domain Search List: ad.demo.co.za; lan.demo.co.za
      • So the thing that I found was: - if Domain Name on VLAN30 was vlan30.demo.co.za I received the following
        • Server: dc.ad.demo.co.za
        • Address: 192.168.30.253
        • *** dc.vlan30.demo.co.za can’t find /dig: Non-existent domain
      • So i have changed the Domain name the dc.ad.demo.co.za, still received an error can’t find /dig: Non-existent domain but it actually allows me to join a domain. I know that its not ideal, but its a few steps closer.
  • So I will be moving Site B to 172.0.21.0/24 as I am running VoIP on 172.0.20.0/28.

  • Only reason why I didn’t have the chance to move them yet was due to printers. I didn’t have access the the printers but after today with some discussion with the printer supplier they allowed me access and now I finally have full control over SiteB network.