DNS Malware Filtering Compared: Quad9 VS Cloudflare VS DNS Filter VS OpenDNS / Cisco Umbrella

quad9dns · May 30, 2020, 4:25pm

Hi - you had asked about a management interface for Quad9. There is no way to monitor Quad9’s filtering via a web interface at Quad9, since we do not have the concept of an “account” due to privacy and data collection issues. However, if you are willing to do a bit of hackery, then it is possible to see which queries are being blocked by our filter versus which queries are being naturally given an NXDOMAIN. We flag our blocked queries with the RD bit in the reply. If you’re using Quad9 as a forwarder, Pi-Hole automatically recognizes this and will flag the blocks appropriately, but it’s possible to just capture the packets manually as well.

If you’re a user of tshark (the command-line version of wireshark for Linux) then you could run a command like this constantly to get a list of blocked hosts. Of course, change the ethernet interface for your local installation, and that interface would need to be able to “see” all the query replies coming from Quad9 towards your host(s). This loops to prevent disk exhaustion since tshark keeps temp files. Use “dig @9.9.9.9 +short A blocked.test.on.quad9.net” to trigger a block result.

while true; do tshark -np -i ens32 -q -c 100000 -E separator=, -E quote=d -T fields -e frame.time_epoch -e dns.qry.name -e ip.src -e ipv6.src -e ip.dst -e ipv6.dst -Y “dns.flags.rcode eq 3 && dns.flags.recavail eq 0” “src net 9.9.9.0/24 or src net 149.112.112.0/24 or src net 149.112.149.0/24 or src net 2620:fe::0/48” 2>/dev/null; done

quad9dns · May 30, 2020, 4:37pm

Hi BangkokBob - We actually have a system in Bangkok at a local IX (Inter-eXchange) called BKNIX (https://www.peeringdb.com/ix/1025) in our Packet Clearing House space (they’re another non-profit that is the primary way we get distributed worldwide.) If your provider doesn’t peer (exchange packets) with us there, then they take the packets wherever they see fit. PCH has an open interconnect policy, meaning they’ll interconnect with any network at no cost as long as they’re on the IX. However, many telcos (they always seem to be telcos) have a somewhat antiquated view of the world and think that interconnecting for better performance is a negative (that is a much longer myth than can be explained here) so they may decide that taking your traffic to an entirely different nation or even continent is the “best” thing to do. Apply pressure locally for your ISP/telco to connect to open exchanges and your performance will improve, not just to Quad9 but to everything. Other organizations may choose to pay or otherwise compensate the ISP/telco to interconnect, but we feel that is a bad precedent for an open and robust Internet, and rewards poor architectural decisions.

elvisimprsntr · May 30, 2020, 6:56pm

Switched from OpenDNS Home Free to Quad9 many years ago since OpenDNS did not support DNSSEC or DNS over TLS. Set up pfSense FW rules to route all DNS queries through UNBOUND. Works perfectly.

@quad9dns Now if we can only get pfSense to adopt DNSCrypt natively.

niko · May 30, 2020, 11:58pm

Any chance you could add cleanbrowsing and webtitan to your test? There’s an older test by Nykolas Z out there that shows cleanbrowsing beating quad9 and opendns on phishing domains. Curious how cleanbrowsing compares for malware domains.

quad9dns · May 31, 2020, 3:06am

elvisimprsntr - Is DNS-over-TLS good enough? That works today with recent releases of pfSense. Just add these lines to the “Custom Options” in your DNS Resolver:

server:
forward-zone:
name: “.”
forward-ssl-upstream: yes
forward-addr: 2620:fe::fe@853
forward-addr: 9.9.9.9@853
forward-addr: 149.112.112.112@853

Don’t worry - having the IPv6 address in there won’t hurt anything; it just gets skipped if you don’t have IPv6.

peter · May 31, 2020, 7:54am

Hi there,

I’m a security researcher at DNSFilter, and I was very interested to see this test. I’ve spent a while looking into it because I was surprised by the results, and had a few comments. I had planned to reply yesterday, but my Saturday went a bit askew so I haven’t had a chance until now.

First of all, full credit for explaining your testing procedure and providing details of your test data and the script run. This is great to see, because as you say in your video it allows others to run things in their own environment.

Unfortunately, I think the test data used wasn’t a great source for malicious domains. The obvious indicator here is that the initial list was ~2200 entries long, but you ended up with only 141 after filtering out domains that no longer resolve.

I had a look further into this, and the source (hxxps://isc.sans.edu/suspicious_domains.html) notes its sources. Here’s a quick rundown from the list on that page:

Malware Domain List. com (hxxp://www.malwaredomainlist.com/updatescsv.php)

only has four entries, two of which aren’t actually domains

Domain Blocklist From Malwaredomains (hxxp://mirror1.malwaredomains.com/files/domains.txt)

malwaredomains.com has been deprecated and not updated for a long time

Threatexpert .com Malicious URLs (hxxps://www.networksec.org/grabbho/block.txt)

completely blank file

Virustotal Domains

no actual URL linked to here, and VirusTotal don’t have a public feed of malicious domains

Checking a copy of the list on GitHub (hxxps://github.com/Ultimate-Hosts-Blacklist/DShield.org-Suspicious-Domain-List-High/commits/master/domains.list) shows that very little has changed in the source list for a long time.

It’s tough to find decent public sources of malware. This is actually part of my job, assessing third-party feeds which we can ingest into our system to help improve our filtering, and we’re in the process of integrating several new feeds right now. We’ve also deprecated several in the past few months, because they’re just not good enough quality - they have a high percentage of false positives, and don’t offer very much protection.

I’ve re-run the test locally, and can already see that even with the remaining domains there are some issues. There’s a couple of very weird results (ie, IPs being different from different resolvers), and a lot of domains that don’t have any accessible services or content anymore. I need to go through the list properly to give a decent report, though.

I would provide a decent public source you could use for domains to use in this test, but I don’t want there to be any suggestion of bias. However, we’d love to work with you to improve the test, because honestly we want the same thing: decent protection from online threats. Please get in touch!

Thanks again for being open about your testing, I really do have to commend your transparency here.

cheers,

Peter

PS: URLs have been lightly obfuscated because there’s a limit of 2 URLs per post for new users. Feel free to edit this post to properly link them up.

mtheofy · May 31, 2020, 8:36am

Hello from Greece. Been following Tom on YouTube but this video gave me a reason to signup on the forum.

I recently switched to Quad9 (57ms latency) from Cloudflare (17ms latency) since the latter was giving me some blank replies on some DNSSEC-enabled zones. I also found that Quad9 is more transparent in their documentation and provide more granularity (e.g. I am using the “Blocklist, DNSSEC, EDNS Client-Subnet sent” service).

I do advocate Quad9 to my peers but the latency discussion is always a show stopper. @quad9dns is there any plans for a closer to the home service (i.e. Easter Europe)? We are currently being served by London.

Just for the fun of it, here’s my forward section of my DNS resolver box (unbound) at home. I forward google and cloudflare related queries to their DNS services and the rest to Quad9

# Google related queries to 8.8.8.8
forward-zone:
 name: "google.com."
 forward-addr: 2001:4860:4860::8888@853
 forward-addr: 8.8.8.8@853
 forward-addr: 2001:4860:4860::8844@853
 forward-tls-upstream: yes

forward-zone:
 name: "google.gr."
 forward-addr: 2001:4860:4860::8888@853
 forward-addr: 8.8.8.8@853
 forward-addr: 2001:4860:4860::8844@853
 forward-tls-upstream: yes

forward-zone:
 name: "youtube.com."
 forward-addr: 2001:4860:4860::8888@853
 forward-addr: 8.8.8.8@853
 forward-addr: 2001:4860:4860::8844@853
 forward-tls-upstream: yes

forward-zone:
 name: "youtube.gr."
 forward-addr: 2001:4860:4860::8888@853
 forward-addr: 8.8.8.8@853
 forward-addr: 2001:4860:4860::8844@853
 forward-tls-upstream: yes

forward-zone:
 name: "like.com."
 forward-addr: 2001:4860:4860::8888@853
 forward-addr: 8.8.8.8@853
 forward-addr: 2001:4860:4860::8844@853
 forward-tls-upstream: yes

forward-zone:
 name: "gmail.com."
 forward-addr: 2001:4860:4860::8888@853
 forward-addr: 8.8.8.8@853
 forward-addr: 2001:4860:4860::8844@853
 forward-tls-upstream: yes

forward-zone:
 name: "googleapis.com."
 forward-addr: 2001:4860:4860::8888@853
 forward-addr: 8.8.8.8@853
 forward-addr: 2001:4860:4860::8844@853
 forward-tls-upstream: yes

forward-zone:
 name: "blogger.com."
 forward-addr: 2001:4860:4860::8888@853
 forward-addr: 8.8.8.8@853
 forward-addr: 2001:4860:4860::8844@853
 forward-tls-upstream: yes

forward-zone:
 name: "android.com."
 forward-addr: 2001:4860:4860::8888@853
 forward-addr: 8.8.8.8@853
 forward-addr: 2001:4860:4860::8844@853
 forward-tls-upstream: yes

forward-zone:
 name: "doubleclick.com."
 forward-addr: 2001:4860:4860::8888@853
 forward-addr: 8.8.8.8@853
 forward-addr: 2001:4860:4860::8844@853
 forward-tls-upstream: yes

forward-zone:
 name: "adwords.com."
 forward-addr: 2001:4860:4860::8888@853
 forward-addr: 8.8.8.8@853
 forward-addr: 2001:4860:4860::8844@853
 forward-tls-upstream: yes

forward-zone:
 name: "googleanalytics.com."
 forward-addr: 2001:4860:4860::8888@853
 forward-addr: 8.8.8.8@853
 forward-addr: 2001:4860:4860::8844@853
 forward-tls-upstream: yes

forward-zone:
 name: "google-analytics.com."
 forward-addr: 2001:4860:4860::8888@853
 forward-addr: 8.8.8.8@853
 forward-addr: 2001:4860:4860::8844@853
 forward-tls-upstream: yes

forward-zone:
 name: "waze.com."
 forward-addr: 2001:4860:4860::8888@853
 forward-addr: 8.8.8.8@853
 forward-addr: 2001:4860:4860::8844@853
 forward-tls-upstream: yes

forward-zone:
 name: "1e100.net."
 forward-addr: 2001:4860:4860::8888@853
 forward-addr: 8.8.8.8@853
 forward-addr: 2001:4860:4860::8844@853
 forward-tls-upstream: yes

forward-zone:
 name: "googleusercontent.com."
 forward-addr: 2001:4860:4860::8888@853
 forward-addr: 8.8.8.8@853
 forward-addr: 2001:4860:4860::8844@853
 forward-tls-upstream: yes

forward-zone:
 name: "g.com."
 forward-addr: 2001:4860:4860::8888@853
 forward-addr: 8.8.8.8@853
 forward-addr: 2001:4860:4860::8844@853
 forward-tls-upstream: yes

forward-zone:
 name: "goo.gl."
 forward-addr: 2001:4860:4860::8888@853
 forward-addr: 8.8.8.8@853
 forward-addr: 2001:4860:4860::8844@853
 forward-tls-upstream: yes

forward-zone:
 name: "ytimg.com."
 forward-addr: 2001:4860:4860::8888@853
 forward-addr: 8.8.8.8@853
 forward-addr: 2001:4860:4860::8844@853
 forward-tls-upstream: yes


forward-zone:
 name: "yt.be."
 forward-addr: 2001:4860:4860::8888@853
 forward-addr: 8.8.8.8@853
 forward-addr: 2001:4860:4860::8844@853
 forward-tls-upstream: yes

forward-zone:
 name: "youtu.be."
 forward-addr: 2001:4860:4860::8888@853
 forward-addr: 8.8.8.8@853
 forward-addr: 2001:4860:4860::8844@853
 forward-tls-upstream: yes

forward-zone:
 name: "google.co.uk."
 forward-addr: 2001:4860:4860::8888@853
 forward-addr: 8.8.8.8@853
 forward-addr: 2001:4860:4860::8844@853
 forward-tls-upstream: yes

# Cloudflare related queries to 1.1.1.1
forward-zone:
 name: "cloudflare.com."
 forward-addr: 2606:4700:4700::1111@853
 forward-addr: 1.1.1.1@853
 forward-addr: 2606:4700:4700::1001@853
 forward-tls-upstream: yes

forward-zone:
 name: "cloudflare.at."
 forward-addr: 2606:4700:4700::1111@853
 forward-addr: 1.1.1.1@853
 forward-addr: 2606:4700:4700::1001@853
 forward-tls-upstream: yes

forward-zone:
 name: "cloudflare.us."
 forward-addr: 2606:4700:4700::1111@853
 forward-addr: 1.1.1.1@853
 forward-addr: 2606:4700:4700::1001@853
 forward-tls-upstream: yes

forward-zone:
 name: "cloudflare.co.at."
 forward-addr: 2606:4700:4700::1111@853
 forward-addr: 1.1.1.1@853
 forward-addr: 2606:4700:4700::1001@853
 forward-tls-upstream: yes

## All the rest to upstream (currently quad9)
forward-zone:
 name: "."
 forward-addr: 2620:fe::11@853 # quad9.net secondary
 forward-addr: 9.9.9.11@853         # quad9.net primary
 forward-addr: 2620:fe::fe:11@853 # quad9.net secondary
 forward-tls-upstream: yes

BangkokBob · May 31, 2020, 10:09am

Hi, many thanks for your answer. I pay my ISP a tidy sum each month so hope I have leverage. Exactly what is it that I would ask them to do? I understand what you have written, but I can’t quite see how I would phrase a request to them. TIA

LTS_Tom · May 31, 2020, 10:24am

Thank you @peter I will reach out directly via email.

anon31769429 · May 31, 2020, 10:25am

Could you post this list, I’m interested.

peter · May 31, 2020, 10:33am

Hey Kolok. Which list do you mean? I posted links to several, but the main source mentioned in the video is this one:

https://isc.sans.edu/suspicious_domains.html

anon31769429 · May 31, 2020, 10:44am

I was referring to this .

Mike-DNSFilter · May 31, 2020, 12:19pm

Hi pavlos,

CTO & Co-founder of DNSFilter here. When you’re not an authorized user of our service, this is the IP we return. Feel free to signup for a 14-day trial, enter your network info to be authorized, and give us a try.

Peter from my team has also replied in this thread. We work hard to have high quality threat sources, along with our own AI and will be posting more about this during the coming week, with regards to what we feel would be a more accurate test of all providers.

It’s definitely difficult to measure threat protections across providers as threats come and go so quickly, most quality lists aren’t public. One approach, as could be seen in this test, is where quad9 seems to be ingesting and blocking anything it can, making it show up top in the results.

We don’t believe in blocking off parts of the Internet (domains) for all eternity, for example, just because 5 years ago someone’s Wordpress blog was compromised and a week later they cleaned it up. We’re always re-evaluating individual domains and the third party feeds we use.

Quad9 is a great service, but we’re typically addressing different audiences, as we have roaming clients, provide logging, and do content filtering in addition to threat protection.

elvisimprsntr · May 31, 2020, 1:48pm

DNS-over-TLS is good enough, and this post seems to indicate better. https://tenta.com/blog/post/2017/12/dns-over-tls-vs-dnscrypt

The only thing I liked about OpenDNS Home Free is the content filtering. I configured pfSense to use DNS-over-TLS with OpenDNS servers which seemed to work, unfortunately OpenDNS (Cisco) doesn’t officially support it. Since malware protection is higher priority, I switched to Quad9, and have been extremely pleased with availability and performance.

P.S. pfSense 2.4.4+ DNS-over-TLS can be configured via the GUI and does not require custom settings. See link at bottom of post. https://www.netgate.com/blog/dns-over-tls-with-pfsense.html

quad9dns · May 31, 2020, 4:46pm

Hi mtheofy - We actually have quite a few locations and servers spread out across Europe; it’s one of our densest geographic regions, because of national borders and density of IX infrastructure. Greece, sadly, is not one of our POPs - yet. But London seems a bit far, I agree. If we’re not immediately nearby, then your provider has much more control over where the handoff is than we do. We have a list of locations where we have systems and interconnections here: https://www.quad9.net/locations Note that your provider would need to peer with Packet Clearing House (PCH - AS42) at one or more of those locations for the traffic to have a chance to be locally served. We are big advocates of dense, localized traffic exchange. Keeping your DNS queries in the same nation in which they were originated improves performance, security, and privacy. However, that requires the cooperation and participation (investment) of local network providers such as telcos and ISPs, so that is sometimes contradictory to their internal goals.

mtheofy · May 31, 2020, 8:49pm

Thanks for the reply John @quad9dns

Will reach out to some connections I still have with GR-IX and a couple of major ISPs. No promises on any quick success though.

Keep up the great work

quad9dns · May 31, 2020, 9:38pm

mtheofy - Thanks! That would be great. You can reference your contacts to the peering data at PCH - they’re our primary partner for getting footprint worldwide. Typically, if they can get connected to the IX, they provide us with a small amount of rackspace and some transit so we can get our services installed. https://www.pch.net/about/peering

woody · June 1, 2020, 1:23pm

Hi, Bob. I’m Bill Woodcock, John’s counterpart at PCH. As he said, if you’re reaching a server 188ms away, rather than the 3ms-5ms you should expect if you were reaching the Bangkok server, it’s because your ISP is choosing to send the traffic that way. Can you post a traceroute so we can see where it’s going? Maybe we can reach out to them and try to get them to fix the problem.

woody · June 1, 2020, 1:29pm

As Jefronty points out, there’s a second IPv4 address, 149.112.112.112, but there’s also a primary and secondary IPv6 address, 2620:FE::FE and 2620:FE::9, and you should definitely configure those as well, since you may have a faster or more reliable path over IPv6 than IPv4 in some cases. May as well take full advantage of it!

woody · June 1, 2020, 1:48pm

As John says, it’s up to your ISP to decide where to send your traffic. You can see the current server clusters here. The closest ones would be Sofia, Tirana, and Istanbul.