Adblock Plus and (a little) more
Blocking malicious sites with Adblock Plus · 2008-07-03 11:48 by Wladimir Palant
Note: Sorry, comments are closed now (2010-09-28). While it is great that lifehacker decided to mention my two years old blog post, I don’t really feel like reading comments from people who didn’t bother to read my blog post. If you need some info on Malware Domains list or want to comment on it — malwaredomains.com is where you should go. If you want to comment on the lifehacker article — please do so on lifehacker.com.
I was reading about yet another wave of attacks exploiting a Flash vulnerability. It turned out that the Flash vulnerability used was already fixed but that doesn’t really matter — Adobe seems incapable of updating users to a secure Flash version in a timely fashion. So Firefox users were at risk here as well, and the continuing waves of SQL Injection attacks inserting malicious iframes into trusted websites didn’t exactly make the situation better.
Yet the domains participating in the attacks are known, so there must be a way to block them. Of course I checked the malware filter in Firefox 3 first, yet it didn’t recognize the sites as malicious. It might have been that the sites were too new, yet the information on Google for some of the older domains indicated that these have been scanned and nothing objectionable could be found. Not sure what Google’s scans look at, but I guess they simply have a different focus — the idea is to block sites the user might go to unintentionally rather than malicious Flash objects that could be easily served with ads for example.
I searched for other lists of malicious domains and quickly found one aggregating multiple sources of information including Dancho Danchev’s blog posts I linked to above. The list is mainly meant for DNS servers, but why not try to use it in Adblock Plus? The script to convert the list was easily written, discussing the matter with the author of the list and finding hosting for the Adblock Plus filter subscription took somewhat longer — but now it is all done.
So now Adblock Plus users can add a subscription with slightly over 40000 filters that will block access to the known malicious domains. It is the first time I tried Adblock Plus with so many filters, and the good news is: the slowdown during browsing is in the area of single-digit millisecond numbers, that’s not noticeable. The bad news: loading/saving the list still takes a while (noticeable as browser startup/shutdown delay). In Firefox 2 this took around 20 seconds which is why I recommend against using this subscription there. The big surprise was Firefox 3, there the delay is only 3-4 seconds. Congratulations to everybody who helped optimizing JavaScript, the results are really incredible!
Using this filter subscription will also require 20 MB more memory and up to 25 MB download bandwidth per month. I’ll continue working on performance optimizations, but if you can live with the performance cost and want to try it already: click here to subscribe to the list in Adblock Plus (listed on the usual list of filter subscriptions as well of course).
Comment [25]
Commenting is closed for this article.
Amsterdammer · 2008-07-03 12:40 · #
Google? Malware filter in Firefox 3? Forget it!
Within the last 3 week I have reported two times via menu>help>report web forgery a website with a malicious code, which is blocked by my Kaspersky. Searching in the virus-encyclopaedia of Kaspersky I found, that this code is well known since 2007 (http://www.viruslist.com/de/analysis?pubid=200883546 (in German), scroll to Nr. 4: iframe Attacken…). But nothing happened the last 3 weeks, site is still not blocked by Firefox 3.
So, who decides for the Fx3 users, what is malicious, what not? Google? IMHO too slow and unreliable..
Fox · 2008-07-03 15:23 · #
why there is / after every domain extension, like .com/
what if some listed domain uses port number, like:
.com:8080 or .com:81
Then that list does not block it.
Reply from Wladimir Palant:
That’s to prevent false positives. So far it seems that port numbers other than 80 are very uncommon – if that assumption turns out to be wrong we can change the list.
Fox · 2008-07-03 16:03 · #
WP: “That’s to prevent false positives.”
And i did just remember:
.com.au
.net.au
.com.sg
-domain extensions and some others,
then that list is better that way.
ecjs · 2008-07-03 19:56 · #
This one is a nice solution too :
http://www.k9webprotection.com/
Asshole · 2008-07-03 23:06 · #
Kudos to you, Mr. Palant, for this Malware Domains list. I have loaded this list and will watch its hit counts with interest.
Asshole · 2008-07-03 23:17 · #
Wait a minute—what is the point of this list? I have it installed and enabled (and ABP is also enabled), yet was able to navigate to one of the domains in the list without (007arcadegames.com) a problem. Surely I’m missing something.
Reply from Wladimir Palant:
It won’t prevent you from going to this page – but it will block for example any iframes or Flash objects from this domain.
Asshole · 2008-07-04 00:21 · #
The thing with K9WP is that its filters are system-wide; they are not user-based . Thus you either block categories for all users or none. The last time I used it there was also a very noticable slowdown for loading new domains, due to the way each domain is checked before being loaded.
If you need multi-user blocking you’re better off using OpenDNS’ content-filtering and adult-site blocking features .
Fury · 2008-07-06 19:05 · #
I think this filterlist is a great idea.
But why is it necessary to download the whole filterlist every time. I think there will be only slight changes to the list. Wouldn’t it be possible to download only the changes and Adblock Plus integrates them locally in the list. This would spare a lot of bandwith and server traffic.
Reply from Wladimir Palant:
Adblock Plus doesn’t support that mode yet – and it is more difficult to implement on the server side. Nevertheless, that’s a direction we might try in future.
Verb · 2008-07-08 14:54 · #
Wladimir, you have a list of 4000 malicious domains. Why not report them to Mozilla to block them instead of having the list on every user’s machine that may reduce the browser speed?
Reply from Wladimir Palant:
I have a list of 20000 malicious domains. And they come from a source that is available to Google as well – problem seems to be really that Google has a different focus with its list, as I said in the article. So reporting those domains is pointless.
hi · 2008-07-11 09:57 · #
Hey Wladimir Palant:
You realize that if everyone on the web used adblock, 99.9% of good websites would either shut down, or become pay sites?
I hope someone robs your home, the way you are helping people rob others.
Please get cancer and die.
RNiK · 2008-07-15 11:25 · #
I have installed and enabled the above list. Then I start noting frequent browers hangs (I’m still using Firefox 2.x) so I looked at firefox.exe process throught Process Explorer and I discovered the problem was the connection with malwaredomains.com and malware.com.
Unfortunately I have to disable the above list.
Reply from Wladimir Palant:
Right, and of course you didn’t read the numerous warnings not to use that list with Firefox 2…
RNiK · 2008-07-15 13:00 · #
…numerous warnings…
…Firefox 2…
…ME = zzzz…
DOH!
Sorry for the spam.
worriedMan · 2008-07-16 23:35 · #
Wladimir,
The good folks at ShadowServer.org are maintaining an up to date list of the urls found on SQL injected pages. It can be found at
http://www.shadowserver.org/wiki/pmwiki.php?n=Calendar.20080514
It’s pretty simple to scrape this text and use a script to convert it into an ABP filter list, and it’s a much shorter and more direct approach to the problem.
Perhaps you could persuade them to publish it in a more acessable form and add it as an alternate subscription?
Reply from Wladimir Palant:
Looks interesting. I’ll contact them about that list – in particular asking whether they themselves think that creating a blocking list from it would be a useful thing. 500 filters is quite a difference compared to 40000 of course.
worriedMan · 2008-07-17 18:22 · #
Wladimir,
As I understand it, this list is only the injected urls observed on SQL injected sites, not the secondary urls in the cascade (mostly redirects which ABP can’t block) and not the ultimate malware dispensaries. These are what actually show up in the injected script/iframe/whatever tags. Hence this list is much shorter and more targeted and can be used to block the entire attack at the source.
The malwaredomains list is much broader and deeper covering all manner of attack sources, destinations and intermediate vectors, hence compartively huge.
I’ve been using this list for about three weeks now and have already registered some hits. It seems to be updated rapidly as each new wave of injections is detected (a practice that we need to encourage). I still have some questions about the best way to organize/format it for ABP, perhaps we should take this discussion to the forum?
Reply from Wladimir Palant:
Yes, the forum is the best place for discussions.
Havvy · 2008-07-24 09:59 · #
Can you add *.on.nimp.org for the malice the site does…no viruses, but you lose your session.
Reply from Wladimir Palant:
I am not maintaining that list, http://www.malwaredomains.com/ is the right place to make suggestions.
papi · 2008-12-23 21:41 · #
Regarding the / after .com, you say it is to reduce false positives and that other than port 80 is uncommon…What about the cases where it still uses port 80 but is just done explicitly, in other words it is for the intended purpose of avoiding block list regex matching?
Reply from Wladimir Palant:
The browser normalized URLs. When Adblock Plus gets the address port 80 is already removed, even if it was in the URL originally.
phil · 2008-12-25 10:33 · #
hi all and merry xmas.
there must have been many performance optimizations done recently as i’ve been using the malwaredomains list on ff2 for a couple of weeks now with no problems at all.
start up time is 10 seconds, shut down time on abp preferences is 3 seconds, shut down time on browser is instant with no ‘hanging’ and no noticeable slowdown in browsing.
i use the fanboy list as well and i can’t thank all involved for looking after the security of non techie guys like me.
fantastic and well done.
regards,
phil.
Reply from Wladimir Palant:
No optimizations from what I can tell – malware domains list is still as huge as ever, and Firefox 2 didn’t get any improvements either (it is a stable branch). You mean that the performance improvements in Adblock Plus 1.0 had such an effect in Firefox 2?
phil · 2008-12-25 14:10 · #
hi Wladimir.
‘You mean that the performance improvements in Adblock Plus 1.0 had such an effect in Firefox 2?’
no, this is the first time i tried this list as i was worried about the warnings not to use it in ff2.
i thought i’d try it anyway and, as i said before, it’s run faultlessly.
btw, should i disable windows defender now i’m using the malwaredomains list? (i never use any other browser)
thanks,
phil.
Reply from Wladimir Palant:
No, you shouldn’t. MalwareDomains list is by no means a complete defense, only another layer of protection.
nimd4 · 2009-02-28 02:25 · #
Malware Domains ( http://malwaredomains.lanik.us/malwaredomains_full.txt ) has the entry:
*.sendspace.com/
…which srews up sendspace.com xD
ps.
Amsterdammer, hopefully things are a little clearer now?..:)
pps.
RNiK · 2008-07-15 11:00 · good correction!
Reply from Wladimir Palant:
Makes sense, I guess sendspace.com is often being used to host malware. It is probably a good idea to add $third-party option to all filters in Malware Domains list occasionally…
linhui · 2009-11-24 08:23 · #
Ugg Australia has recruited designers like Manolo Blahnik, Carlos Falchi, Rebecca Minkoff, Rafe pictured,and more to design high-end versions of the brand’s trademark boots for its seventh annual Art & Sole auction.<br><br><br>I have to assume these will be filed under unaffordable (which is good – more money for St. Jude!), but I’ll still be going to the <a href=“http://www.pickyouruggs.com//”>pickyouruggs.com</a> Web site on December 1 to window shop and see what these designers have cooked up.<br><br>I am trying to picture what I would do to make these high end… Real Italian leather? A pearl brooch? Other expensive ingredients and sparkles? Maybe the differences won’t be that extreme, but I hope each designer puts his personal touch on the shoe. I’m certain at the very least you’ll be able to pick out which Uggs came from Betsey Johnson (<a href=“http://www.pickyouruggs.com//”>UGG Classic tall 5815 boots</a>, please!).<br><br>The event is likely to inspire as many interesting Ugg knockoffs as there are regular Ugg knockoffs, so it is possible we’ll see something like this around town next year. It’s not only a great event for charity but probably also a good move to keep Uggs alive and relevant… especially with the number of people who’d like to see them disappear for good!<a href=“http://www.pickyouruggs.com//”>UGG boots on sale</a>!<br><br>I will of course keep wearing my Uggs because A) I have them and B) they are warm, but if your kid asks Santa for Where the Wild Things Are <a href=“http://www.pickyouruggs.com//”>UGG boots</a>… I feel for ya.
mfkr · 2009-12-15 07:42 · #
Hi, I want to know how to update the filter subscription: Malware domains. when I right click and update the subscription it says: failed, checksum mismatch…
When I open this list (this is where it updates) http://malwaredomains.lanik.us/malwaredomains_full.txt why it says in the top [Adblock Plus 0.7.2] when the last version and the one that i have installed is 1.1.2
Reply from Wladimir Palant:
Yes, checksum is calculated incorrectly. I sent the server owner a corrected version of the script generating that list – once it is replaced on the server things should go back to normal.
ibrahimali · 2010-01-04 15:34 · #
PLEASE DOWNLOAD FROM WEBSITE www.goldenduas.com for peace, security, health and wealth for man kind. The same may be circulated for all concern based on Quranic verses 2:2,10:57,17:11 and 16, 28:59,39:55 and 57 and also 65:8
Warm regards
Ibrahim Ali
CH · 2010-05-15 09:30 · #
THANK YOU for this, you just saved me a million headaches!
Wes W. · 2010-09-28 16:01 · #
Thanks for making the filter list, however, using a service such as OpenDNS won’t use any extra memory or resources and actually speeds up your browsing – while still providing excellent protection. Knowing this, I fail to see the benefit of this ABP filter list.
BSkiLLs · 2010-09-28 17:30 · #
So seems to be a good list but…. How did you get some of them as some are normal sites it seems. Like yourdesignart.com
What i mean is just cause somebody may of had a few malware on there site before like by mistake and was told & removed it etc… How do you keep them all up to date and know for sure they are bad and ALWAYS bad ?? Cause nothing against you of course but i ‘know’ you don’t go through every site personally and go to visit the sites mthly or even yearly. As that’s alot to go threw but…
So yeah what’s the deal there ?????
[i have no clue whose site that is or what it’s about just using the domain as an example cause name seems like its normal site. Enough to get point across as if its just a design site or blog etc..]
Thxs.