Hi there,
I’m a security researcher at DNSFilter, and I was very interested to see this test. I’ve spent a while looking into it because I was surprised by the results, and had a few comments. I had planned to reply yesterday, but my Saturday went a bit askew so I haven’t had a chance until now.
First of all, full credit for explaining your testing procedure and providing details of your test data and the script run. This is great to see, because as you say in your video it allows others to run things in their own environment.
Unfortunately, I think the test data used wasn’t a great source for malicious domains. The obvious indicator here is that the initial list was ~2200 entries long, but you ended up with only 141 after filtering out domains that no longer resolve.
I had a look further into this, and the source (hxxps://isc.sans.edu/suspicious_domains.html) notes its sources. Here’s a quick rundown from the list on that page:
Malware Domain List. com (hxxp://www.malwaredomainlist.com/updatescsv.php)
- only has four entries, two of which aren’t actually domains
Domain Blocklist From Malwaredomains (hxxp://mirror1.malwaredomains.com/files/domains.txt)
- malwaredomains.com has been deprecated and not updated for a long time
Threatexpert .com Malicious URLs (hxxps://www.networksec.org/grabbho/block.txt)
- completely blank file
Virustotal Domains
- no actual URL linked to here, and VirusTotal don’t have a public feed of malicious domains
Checking a copy of the list on GitHub (hxxps://github.com/Ultimate-Hosts-Blacklist/DShield.org-Suspicious-Domain-List-High/commits/master/domains.list) shows that very little has changed in the source list for a long time.
It’s tough to find decent public sources of malware. This is actually part of my job, assessing third-party feeds which we can ingest into our system to help improve our filtering, and we’re in the process of integrating several new feeds right now. We’ve also deprecated several in the past few months, because they’re just not good enough quality - they have a high percentage of false positives, and don’t offer very much protection.
I’ve re-run the test locally, and can already see that even with the remaining domains there are some issues. There’s a couple of very weird results (ie, IPs being different from different resolvers), and a lot of domains that don’t have any accessible services or content anymore. I need to go through the list properly to give a decent report, though.
I would provide a decent public source you could use for domains to use in this test, but I don’t want there to be any suggestion of bias. However, we’d love to work with you to improve the test, because honestly we want the same thing: decent protection from online threats. Please get in touch!
Thanks again for being open about your testing, I really do have to commend your transparency here.
cheers,
- Peter
PS: URLs have been lightly obfuscated because there’s a limit of 2 URLs per post for new users. Feel free to edit this post to properly link them up.