Were are the(se) unknow dns requests coming from?

I use pfSense latest stable router / firewall. I have in pfblocker-ng set PRI1 blocklist. I also use DNSBL with several lists:


I also have Snort enabled.

Since few months i see strange activity in the firewall logs i can’t figure out were they are coming from.
I blurred my own public ip.
This is some traffic that is leaving my WAN. What is triggering this? How can i find out what is triggering this? I have 7 vlan interfaces i have viewed them all with wireshark to see what happen and filtered on the range 193.163.125.0/24

The ip range 193.163.125.0/24 is on a block list Snort is using so snort blocks this range but why are ip’s from this range triggering a dns respons on my pfSense?

I figured something is trying to connect to a domain that needs looked up on a dns server on the 193.163.125.254 dns server.

afbeelding

Who can shine a light on this?

You can use pftop to trace what is calling out to that IP address.

1 Like

Thank you Tom, I watched the video and have pfTop open for 2 hours till the DNS requests showed op again. But nothing showed up in pfTop. For some reason pfTop is not picking this up :man_shrugging:

Then do a full packet capture until you find it.

1 Like

With that video I monitored all my 7 vlans and the WAN interface.

What I see is on the WAN many ip’s in the 193.163.125.0/24 range coming in on the WAN tcp sync (see the pics above).

Somehow this triggers a pfSense unbound outgoing connection to a remote dns server on 193.163.125.254. How do they do that? Because this way they could DOS my unbound dns server on pfsense.

Not sure, try posting in the pfsense forums. They already have a similar topic form a few year ago.

I know about this post on the pfSense forums there is also another one about this subject but not exactly the same problem as i have, i think. I can change settings in dns resolver without any problems and i didn’t change anything manually in the system either.

For a long time I am trying to figure this out, no luck yet.
If I ever find out what this is i will come back and let the board know.

Thanks a lot anyway Tom :facepunch:

1 Like

@OP, first thing, from what VLAN are there requests coming from? Second, what devices are on that network? Do you have any IoT devices like intelligent TVs, cameras, printers, video recorder, fridge, etc that could try to phone home? The IP you put forth 93.163.125.254 is from Denmark, so it must be related to some new devices or softwares you installed lately if these were not showing before.
Here you can find a list of public DNS of Denmark (DNS servers in Denmark) but that IP isn’t showing as a registered name server.
The reverse lookup of that IP is 93-163-125-254-static.dk.customer.tdc.net, which seems to belong to tdc.net who is an ISP by all accounts.
After that, you need to use other tools to get more information.

All this happens on the WAN wich is not a vlan but a hardware nic.
What happens is: every 20-60 minutes a connection from the 193.163.125.0/24 range is coming in on the WAN. Somehow these connections trigger a DNS lookup to their 193.163.125.254 DNS server. Both actions (incoming and outgoing) are blocked by Snort and a pfblocker blocklist. These connections are coming from some cyber.casa security research company. On their website they write that they scan the internet for internet faced services. Since Snort is in legacy mode i gues they sometimes get thru and hit one of my servers i run on my network.

Many other scans and connections happens every second of the day as you might know but this one stands out because these connections coming from this 193.163.125.0/24 range trigger a dns lookup by my pfsense unbound server (dns resolver). So in theory they could DOS my dns resolver service on the pfsense box.

What i don’t understand is how do they trigger this unbound dns lookup to their dns server on 193.163.125.254 with a single tcp:s connection on my WAN?

I recall also that when I was using pfSense, I had similar traces in my logs about “unwanted” incoming traffic of something that orinated from my network (after investigation) and I was puzzled too - and a little panicked until I understood what was at scheme.

What I found was that a machine on a certain vlan that had to vpn out to another site for tele-working was causing this. Some were replies to whatever initiated traffic on the machine that went through the vpn but came back directly to pfsense WAN IP instead of the tunnel IP of the machine, others were genuine initiated communications to my WAN IP (they were from the same foreign IP).

After digging on that tele-working machine, a few questions asked to its owner and their network support, and some packet capture, I found that some requests going thought the vpn included my public IP somehow, and were passed to a tiered destination who would in turn try to reply back directly to my WAN IP instead of going back through the vpn and reply to the application on the tele-working machine.

Seems to me as bad routing of replies on the other side of the vpn (which I don’t control) of their tiered services. Since that tele-working machine was already in its own vlan and isolated from the rest of the other networks, and once I understood what was happening, I stopped caring about this as pfSense & cie were doing their normal job and I wasn’t getting hacked or something.

So maybe you are experimenting this kind of behavior too and those logs are badly routed replies from whatever services out there.

Sure could be something on my own network causing this. I am not using any outgoing VPN. I do have a openVPN server running on pfsense for myself only to login to my network wen on the road. I don’t use outgoing VPN so that could not be the case. Btw these connections i am talking about are going on 24/7 for many months if not years.

I monitored for hours and hours my local networks (7 vlans) for connection / traffic coming from or going to 193.163.125.0/24. I think if this was the case i could have picked them up with pfTop like Tom talked about in his video. This seems solely to be going on on the WAN nothing on the inside.

If i would use some kind of tunnel with something I could imagine a scenario like you experienced.

I do have some IOT devices they have no internet access at all. They are on a separate vlan of their own. But still i also monitored this IOT vlan extensively because i am determined to figure this out.

Probably it is something really simple it always is but what is it? :grinning:

Thank you for your post and ideas about this. :+1:

It’s possible that it could be coming from pfSense (maybe a package?). I would double check your unbound settings (Services > DNS Forwarder and Resolver, forwarder service off, resolver service on, check all resolver settings) as well as the system DNS settings (System > General Setup). Maybe take a look at Dynamic DNS settings too for good measure. Sometimes you just have to go through all of the settings one screen at a time. Also, If you create a floating firewall rule to block that 193.163.125.0/24 subnet on all interfaces and enable logging maybe you will get more information. Just a thought.

Since pfBlocker is blocking the initial incoming tcp:s connection maybe pfBlocker is doing the outgoing DNS lookup. I don’t know how to figure that out. The outgoing DNS lookups are blocked by Snort as you can see here Were are the(se) unknow dns requests coming from? - #9 by Gerard

Adding a floating firewall rule on top is a good idea. However, in pfBlocker i configured pfblocker to use floating rules. Wen i add a floating rule on top pfblocker will change the rules order pfblocker auto rules always back to the top.

In system > General > Setup i have no DNS servers configured the pfSense system uses DNS Resolver for all DNS lookup of the network.

The forwarding option of the DNS resolver is not enabled.

In DDNS i have configured my domain with a Canadian DNS registrar / DNS provider. I am using this DDNS for my domain many years, long before these WAN connections taked place and also long before i even started using pfSense.

You come up with several very good ideas were to look and to check but nothing i can connect with this problem.

The only thing i can think of is this organisation has crafted some TCP:S packet that triggers this outgoing DNS lookup to a DNS server on their network.

My WAN connection receives many many connections per second like with every WAN of everybody ofcourse but this incoming connection is the only one that triggers this outgoing DNS connection to their DNS server. This is very smart of them but how do they do it thats what is puzzeling me for a long time. And besides of that i am concerned they or somebody else could DOS the unbound server on de pfsense box.

The idea of the floating rule was simply a means of monitoring all of the internal interfaces without having to spend a lot of time running packet captures. I personally drop all incoming by default and do not log this as it’s all noise with nothing actionable. It’s good to monitor the outgoing traffic but I highly doubt this is something being triggered by the outside IP and it’s likely a checkbox somewhere. A firewall rule was just a means of figuring out where. Speaking of which…in pfSense under Services > DNS Resolver what do your interfaces look like? The outgoing is WAN and the network interfaces are your LAN , VLANs, and localhost?

Yes WAN and DMZ are in the outgoing interfaces of dns resolver. Also DMZ because I have a Bind9 authoritative DNS server for my local domain running on the DMZ network.

Incoming I have all the local IPv4 networks selected.

afbeelding

The floating rule is a smart idea because its before the interface rules order and if it would catch some connection of the 193.163.125.0/24 range i would see it in de firewall logs. I understand what you are trying to say. The auto rules of pfblocker are putting themselves on top of the manual floating rule i added so i need to disable pfblocker to see the log entries of the manual floating rule. It makes no sense though because we already know the manual floating rule would catch the incoming 193.163.125.0/24 connections because pfBlocker reports that in the firewall log.

afbeelding

I only log blocked traffic btw.

Are your pfBlocker floating rules applied to all interfaces? If so then you could just add a rule within pfBlocker to block that subnet and log the traffic. You can then move it to the top of the pfBlocker list and see what it catches. If this is already the case and it only sees traffic on the WAN interface then I would expect the traffic to be coming from within pfSense somewhere, although as already mentioned is possible for an encrypted tunnel to be offloading something. I would also take a close look at whatever is in your DMZ. Does bind send requests to unbound on pfSense? Does bind allow requests from the outside (I assume yes on this one)? Do you have any ACLs setup in your pfSense DNS server? At this point I would probably just setup a rule at the top of pfBlocker to block this subnet and not log the traffic because, while a nice puzzle, it doesn’t seem to be an easy fix and fixing it is not terribly productive (I do this to keep my logs clean. Yes I know X thing is bad and don’t need to hear about it.). Have you thought about taking a backup of pfSense and loading it into a virtual machine and trying to replicate the issue?

Also, have you tried adding “cyber.casa” to the pfBlocker > DNSBL > TLD Blacklist/Whitelist > TLD Blacklist list? This might help catch something on the inside trying to lookup a domain that wont get resolved because you block it.

“Are your pfblocker floating rules applied to all interfaces?”
The WAN is set as the incoming and for the outgoing all interfaces are selected in pfBlocker so yes.

Oke that is a good idea to add the ip range in pfblocker and put it on top of everything. So i just did that a minute ago i added the ASN to a custom ip block list and put that on top of the ip lists. And enabled the log function in that list

“If this is already the case …” Well the range 193.163.125.0/24 is already in one of the pfBlocker (PRI1) list but i put a new custom blocklist above that and all other lists just for this range on itself and block both incoming and outgoing and enabled logging. So I will see what this is going te let me see in de logs.

My domain on the internet is hosted on a dns server of a dns provider. My local domain is hosted on my own bind9 server. My local bind9 in dmz is not reachable from the internet. I have in pfSense unbound a domein override for my local domain so ubound talks to bind9 in dmz wen is needs to lookup my local domain. My local bind server does not talk to outside servers no forwarders nothing it is solely authoritative server for my local domain nothing else. I am not using it as caching server to query any internet domain or whatever all that happens on the pfSense unbound (dns resolver).

Even if bind would want to connect something on the outside it could not get out because pfSense is blocking all outgoing traffic on the DMZ interface.

I have monitored every interface of pfSense and also DMZ with wireshark for any incoming or outgoing connections from or to 193.163.125.0/24 and also monitored the .casa tld. I suspected RSPAMD could be the culprit because rspamd hit a lot of domains wen it updates its database but i find nothing wen i monitor it with wireshark for hours while the cyber.case is still doing what is does on the wan every 20-60 minutes. I don’t see nothing on the dmz interface and nothing on any other interface either and i monitored every interface for hours to be sure i could not have missed anything.

Besides of all this after all this monitoring i also used Tom’s tip to use pfTop. With pfTop nothing shows up while cyber.case is doing what it does on the WAN. So a lot of questions marks here as you can imagine.

An encrypted tunnel? I could not think of where and how this would exist on my network. And could hide itself this well but who knows. Maybe some day i find something somebody hacked my network or something i don’t know i would be hugely amazed if i would discover that.

I have no ACL’s on unbound.

Yes i even have thought of it to create a pfSense system on my proxmox server so i did and still have it for backup or testing stuff.

After creating this virtual pfSense machine and started setting everything up i immediately saw the cyber.casa ip’s showing up again. I didn’t use a config backup i started with a clean empty pfsense vm and started setting everyting up from the gound up, there was no change with the bare metal pfSense.

You saw the IP in the virtual test pfSense or on the main one? When you use Wireshark are you capturing everything on the interface and filtering it later? This may be difficult (I would use several 100 Mb files in a ring buffer then try and stop it once I saw something in the firewall log) but I am wondering if you will see pfSense querying the root DNS servers and working it’s way down and finally getting blocked at cyber.casa. Thus using the TLB Blacklist on all interfaces to block “cyber.casa” or even just “casa” (there should be no leading dot) is just another way to track down where this is originating from.

I see the cyber.casa on both pfSense systems exactly the same behavior.

I set a custom cyber.casa block list in pfBlocker on top of all other blocklist. The problem with this is that in pfblocker geoip and then TopSpammers list is still automatically put on top in the floating rules and cyber.casa range is in this TopSpammers list. And what i also see is that outgoing Snort blocks it before the pfblocker casa blocklist also so i think snort is in front of pfb on the outgoing but in the incoming pfb is in front of snort that seems weird to me

You can see it in this picture the first line from the bottom is were the tcp:s comes in at 15:03:01 and is blocked by pfB_Top_Spammers list and trigger all the DNS lookups above that at 15:03:02.

I saw this behavoir on my main bare metal pfSense but also on the virtual one exactly the same. The moment i enable pfBlocker and Snort in the pfSense VM i see right away these cyber.casa connections every 20-60 minutes.

With Wireshark i do what Tom is doing in this video of his:

In the past i have tested many things. I filtered on the tcpdump part the network range 193.163.125.0/24 and also on cyber.casa and also on the word casa by itself. I also captured everything and yes you get big files and yes i splitted them in 25mb files wile capturing.

I have a script in Kali with many filters and interfaces and and and stuff i tested in de last year to figure things out. To many things i did and tested. I figured out were this .casa tld is hosted who owns it etc etc

afbeelding

afbeelding

I checked everything out i could think of.

And i don’t care who is scanning or trying to connect to my WAN the concern i have is that this incoming tcp:s connections coming from this 193.163.125.0/24 range is activating / triggering these dns lookups. That is puzzeling me. I can’t think of anything else then they somehow do this within this first tcp:s connection because the next second several dns lookups show up to their DNS server.