Today, we’ve released Adblock Plus 3.0 for Firefox, our first Firefox release based on Firefox’s new WebExtensions rules.
Aside from all the things under the hood, you will immediately notice a few differences in the new ABP for Firefox. First and foremost, it will just look different; those who also use ABP for Chrome or Opera will notice some aesthetic similarities, for sure. Otherwise, you’ll probably pick out the following:
Adblock Plus worked hard to release our 3.0 browser extension for Firefox early because all Firefox add-ons have to convert to the new WebExtensions API by the time Mozilla releases Firefox 57 later in the month. This is not even to mention those already running the development build of 57, on which the old extensions API does not work. Given that, there will be a few features that longtime Adblock Plus for Firefox users will miss in the new release. Rest assured that we’re working as hard as we can to bring as many features as possible to ABP using the new WebExtensions API for Firefox.
Cheers to all the Firefox development going on right now!
]]>At some point, Web Extensions are supposed to become a new standard for creating browser extensions. The goal is writing extensions in such a way that they could run on any browser without any or only with minimal modifications. Mozilla and Microsoft are pursuing standardization of Web Extensions based on Google Chrome APIs. And Google? Well, they aren’t interested. Why should they be, if they already established themselves as an extension market leader and made everybody copy their approach.
It isn’t obvious at this point how Web Extensions will develop. The lack of interest from Google isn’t the only issue here; so far the implementation of Web Extensions in Mozilla Firefox and Microsoft Edge shows very significant differences as well. It is worth noting that Web Extensions are necessarily less powerful than the classic Firefox extensions, even though many shortcomings can probably be addressed. Also, my personal view is that the differences between browsers are either going to result in more or less subtle incompatibilities or in an API which is limited by the lowest common denominator of all browsers and not good enough for anybody.
Because we have no other choice. Mozilla’s current plan is that Firefox 57 (scheduled for release on November 14, 2017) will no longer load classic extensions, and only Web Extensions are allowed to continue working. So we have to replace the current Adblock Plus by a Web Extension by then or ideally even by the time Firefox 57 is published as a beta version. Otherwise Adblock Plus will simply stop working for the majority of our users.
Mind you, there is no question why Mozilla is striving to stop supporting classic extensions. Due to their deep integration in the browser, classic extensions are more likely to break browser functionality or to cause performance issues. They’ve also been delaying important Firefox improvements due to compatibility concerns. This doesn’t change the fact that this transition is very painful for extension developers, and many existing extensions won’t take this hurdle. Furthermore, it would have been better if the designated successor of the classic extension platform were more mature by the time everybody is forced to rewrite their code.
Originally, we hoped to port Adblock Plus for Firefox properly. While using Adblock Plus for Chrome as a starting point would require far less effort, this extension also has much less functionality compared to Adblock Plus for Firefox. Also, when developing for Chrome we had to make many questionable compromises that we hoped to avoid with Firefox.
Unfortunately, this plan didn’t work out. Adblock Plus for Firefox is a large codebase and rewriting it all at once without introducing lots of bugs is unrealistic. The proposed solution for a gradual migration doesn’t work for us, however, due to its asynchronous communication protocols. So we are using this approach to start data migration now, but otherwise we have to cut our losses.
Instead, we are using Adblock Plus for Chrome as a starting point, and improving it to address the functionality gap as much as possible before we release this version for all our Firefox users. For the UI this means:
If you are really adventurous you can install a current development build here. There is still much work ahead however.
The deadline only affects Firefox Desktop for now; in other applications classic extensions will still work. However, it currently looks like by Firefox 57 the Web Extensions support in Firefox Mobile will be sufficient to release a Web Extension there at the same time. If not, we still have the option to stick with our classic extension on Android. Update (2017-05-18): Mozilla announced that Firefox Mobile will drop support for classic extensions at the same time as Firefox Desktop. So the option to keep our classic extension there doesn’t exist, we’ll have to make with whatever Web Extensions APIs are available.
As to SeaMonkey and Thunderbird, things aren’t looking well there. It’s doubtful that these will have noteworthy Web Extensions support by November. In fact, it’s not even clear whether they plan to support Web Extensions at all. And unlike with Firefox Mobile, we cannot publish a different build for them (Addons.Mozilla.Org only allows different builds per operating system, not per application). So our users on SeaMonkey and Thunderbird will be stuck with an outdated Adblock Plus version.
Sadly, we don’t have the resources to rewrite these extensions. We just released Element Hiding Helper 1.4, and it will most likely remain as the last Element Hiding Helper release. There are plans to integrate some comparable functionality into Adblock Plus, but it’s not clear at this point when and how it will happen.
]]>Because of that, a few weeks ago we moved our development builds to AMO to have them signed automatically. You can also download them from AMO directly (see Development Channel section in the extension description). Anybody who installed development builds from our website should be upgraded to the AMO-hosted builds automatically.
It took us a while to get there, mostly because we couldn’t see any good options. Our development builds are created automatically and we didn’t want to start uploading them to AMO manually. AMO thinks about providing a proper upload API but there hasn’t been much progress in this area yet. So rather than waiting further we decided to instrument the existing web interface — not a terribly elegant or reliable solution but it works for now.
Also, the way AMO presented the development channel wasn’t optimal. It was impossible to link to this section directly, so the instructions would have to go like this: “open page, scroll to the bottom, click on Development Channel to expand.” Fortunately, AMO is open-source, so we helped resolve the issues, our changes are live as of yesterday.
We still have two extensions which are meant for internal use and aren’t hosted on AMO: Diagnostics for Adblock Plus and Adblock Plus Unit Tests. It looks like we will start hosting Diagnostics on AMO so that development builds can be supported there the same way as for the other extensions. As to Adblock Plus Unit Tests, the current consideration is to stop providing development builds here — developers who need to test their changes can usually create their own builds.
]]>Because of that, a few weeks ago we moved our development builds to AMO to have them signed automatically. You can also download them from AMO directly (see Development Channel section in the extension description). Anybody who installed development builds from our website should be upgraded to the AMO-hosted builds automatically.
]]>Now, there are really two very different issues mentioned there. One is caused by very non-obvious behavior in Firefox: while Adblock Plus registers a single stylesheet for its element hiding feature, what happens behind the scenes is Firefox creating a new copy of it for each page being loaded (bug 988266). The memory consumption of all these copies can be very significant, like the 2 GB mentioned above for an edge case.
There is some ongoing discussion in this bug, which will hopefully result in a way to remove the duplication in Firefox and have most of the stylesheet data shared between all pages. Beyond that we are working on a feature to let users send us their filter hit statistics (optional and opt-in of course). This data should help filter authors see which filters are actually being used and which can be removed. Reducing the number of filters will also reduce the memory consumption of that stylesheet.
The other issue is the memory consumption of the data structures created by Adblock Plus itself, these are mostly required in order to manage and apply its filters. Current filter lists for Adblock Plus have around 50 thousand filters which (along with supplemental data like filter hits) require around 60 MB of memory. Clearly, that data is stored in a less than optimal way but apparently that’s hard to avoid when working with complicated JavaScript objects.
Of course we are working on identifying unnecessary memory consumption and getting rid of it (see issue 354 for example), but the potential is limited here as long as we stick to JavaScript objects. So instead we want to implement our own way to store data, a project that will hopefully be finished soon. That approach should also improve performance in the long term though it’s not quite clear yet how well the new code will perform.
]]>Have you found a website with broken links? You can report it in the EasyList/EasyPrivacy forum (no registration required). The filter authors will add a new filter to block Adobe SiteCatalyst on this website. Unfortunately, blocking it on all websites by a generic filter isn’t possible because the website owners tend to install it under different names.
You might also be able to block that script yourself. In Firefox you can click the Adblock Plus icon on the problematic web page and choose “Open blockable items” from the menu. Enter “script” as the search term to list scripts only. Typically, the SiteCatalyst script will have “omniture” somewhere in its name, or it will be called “s_code.js”. If the corresponding line isn’t red then the script isn’t blocked — you can double-click the line to open the filter assistant and block it. Make sure to select the first option in the “Look for pattern” section of the filter assistant — you want to block only this one script, not the entire folder.
If you are a website owner using SiteCatalyst, you can disable link tracking in the configuration of your SiteCatalyst instance with the following statement:
s.useForcedLinkTracking=false;
Yes, we did. We contacted Adobe mid-June, listed the bugs in SiteCatalyst and how they can be fixed. We were then asked for exact steps to reproduce and we delivered those, an explanation of how the issue can be reproduced on adobe.com. Our contact confirmed that this was exactly what they needed — and then there was silence. We’ve sent a reminder three months later but the current state is still that Adobe is for some reason unable or unwilling to fix the bugs in their scripts. So Adobe SiteCatalyst continues to cause issues on websites of Adobe’s paying customers and even on Adobe’s own website.
No, fixing these bugs is trivial. The main issue concerns the way data is transmitted. When a link is clicked on the website SiteCatalyst “swallows” the click and creates an image instead to send tracking data. It then generates a fake link click when this image loads. The code looks like this:
if(!im)
im=s.wd[imn]=new Image;
im.s_l=0;
im.onload=new Function('e','...');
The issue here is that the image can fail to load, something that might be caused by Adblock Plus but also by connectivity issues or a server failure. There will be no load
event in that scenario but rather an error
event and so the fake link click will never happen. The solution is to add the handler for both events so that it is always executed when the request is done:
im.onload=im.onerror=new Function('e','...');
Now SiteCatalyst isn’t meant to track more than one link click. Normally only the first link click should be swallowed by the bug above, the subsequent clicks would be handled as intended. The code looks like this:
if(!s.useForcedLinkTracking)
...
else
s.b.removeEventListener(\"click\",s.bc,false);
This should remove the click listener after it is triggered for the first time. The problem is that the code adding the click listener looks like this:
s.b.addEventListener('click',s.bc,true)
Can you spot the bug? Right, the third parameter of the two calls (useCapture) isn’t identical — a capturing event listener has been added but the code tries to remove a non-capturing listener. As a result, removing the listener doesn’t actually happen.
This isn’t really rocket science. The bugs can be fixed with minimal changes that wouldn’t take any developer more than an hour to implement and test. The bug report we sent to Adobe had the same level of detail, it outlined exactly what went wrong and how it can be fixed. Why wasn’t Adobe able to fix their bugs then? Beats me.
]]>There is one issue with relying on community-supplied filter lists in Adblock Plus: these lists are sometimes hosted on unreliable services that will go down without any prior notice. That’s why a fallback solution had to be designed in the early stages of the projects: if a client cannot reach a filter download server several times in a row it should query a fallback URL which could reply with a new location for that filter list. These fallback requests can also be used to notice issues with filter downloads that the owners of the filter lists didn’t notice themselves.
While that mechanism proved very useful over the years, one issue wasn’t anticipated originally: the fallback URL was being triggered constantly, even for filter lists that were accessible just fine. Some people consistently failed to download filter lists (while still being able to access the fallback URL), be it because of strange misconfigurations, malfunctioning security software or weird proxy servers. The range of possible issues increased significantly once all Adblock Plus requests switched to SSL.
The numbers were still pretty low however — until mid-August when we’ve reconfigured easylist-downloads.adblockplus.org to use the recommended SSL settings (disabled SSLv3 and weak cyphers). All the sudden the prevalent reported error became SSL_ERROR_NO_CYPHER_OVERLAP
— it seems that at least several thousands Firefox users cannot use any strong cyphers.
Obvious suspicion would be extremely outdated clients. And indeed, many of these fallback requests come from outdated Firefox and Adblock Plus versions. However, they are not that outdated, mostly at least Firefox 17 and sometimes even the current Firefox 24. The outdated versions seem to be rather a consequence of the same SSL issues, I guess that they prevent these clients from connecting to Mozilla’s update servers as well.
A glance at about:config
offers another possible explanation: under security.ssl3
one can switch off specific cyphers. I cannot really imagine somebody switching off all strong cyphers however, whether intentionally or by accident. Data corruption causing this issue is IMHO too unlikely to be responsible for thousands of misconfigured clients. Malice is however still an option: given the suspicion that NSA conducts SSL downgrade attacks, the idea that settings for some clients could be manipulated to use only insecure protocols while keeping appearances of secure communication isn’t too far-fetched. Of course, changing browser configuration requires the system to be compromised in the first place so this approach only makes sense for a stealth operation where no software should be installed permanently on the system.
I’m normally not into paranoia so I am looking forward to more reasonable explanations. Did I miss something obvious?
]]>There is one issue with relying on community-supplied filter lists in Adblock Plus: these lists are sometimes hosted on unreliable services that will go down without any prior notice. That’s why a fallback solution had to be designed in the early stages of the projects: if a client cannot reach a filter download server several times in a row it should query a fallback URL which could reply with a new location for that filter list. These fallback requests can also be used to notice issues with filter downloads that the owners of the filter lists didn’t notice themselves.
]]>So how many is 200 million downloads exactly? Doing some basic calculations, this means that since the introduction in 2006, the Adblock Plus was downloaded on average approximately once every second… Consistently during the period of over 7 years!
We are constantly working hard to develop the extension further to ensure a flawless web experience on multiple platforms. We would like to take this opportunity to thank our users (each one of you!) for using Adblock Plus, our vast user base of tens of millions users have actively contributed to our success. A final shout out goes to our contributors, the beating heart of the ABP functionality. We thank you for all the voluntary hard work over the years — without you, we would have never achieved such a success.
]]>Babelzilla helped us a lot, managing more than 40 translations for Adblock Plus alone wouldn’t have been possible without it. Yet, I’ve been dissatisfied with the service for quite some time (mentioned it first time two years ago). Reasons are:
it
and it-IT
as the locale code, a “feature” that caused lots of confusion among translators with duplicate translations created all the time.I’ve looked at lots of different options to manage our translations. There was a thought to use Anwiki but it has lots of issues of its own. Pootle was a consideration but it didn’t look like a great solution (e.g. very much geared towards .po
files) and we would still have to host it ourselves.
For a while it looked like Tim Babych’s Adofex would be the way to go, especially after Babelzilla announced plans to move to that platform. So early this year I approached Babelzilla admins with some thoughts on how to make that migration progress faster and generally make Babelzilla a more reliable platform. After not hearing back I finally realized that depending on somebody’s hobby project with something as important as localization might not have been the best idea.
I came across Crowdin in July thanks to a hint from Thomas Greiner. I decided to try translating the Customizations extension there, one that wasn’t on Babelzilla simply because I was afraid of uploading it there due to all the bugs. Here is my experience so far:
Crowdin’s support for the DTD and properties file formats isn’t ideal, they manage to extract the strings from these files all right but not the comments. This is not really surprising given that the way how comments in these formats should be matched to strings is “underspecified”. Fortunately, Crowdin supports a number of other file formats as well. So my solution was to convert all locale files to the Chrome-style JSON format on upload and back on download. This format has the advantage of a well-specified description field and it is generally a lot easier to parse due to the availability of good JSON parsers for basically any programming language.
Another major issue are access keys. Crowdin doesn’t always display the strings in the order in which they are listed in the file, in particular untranslated strings are always listed first. This has the side-effect that locating the label for an access key is sometimes non-trivial. Which is of course a side-effect of translating labels and their access keys as separate strings — not a good idea in the first place. I consider merging them into one string for the translations in future.
There are also some minor annoyances like there being a limited number of supported languages (I requested adding all the additional languages supported by Firefox but that appears to be non-trivial). Or the not quite yet mature placeholders support. Or the overly crowded (pun intended) translation interface.
]]>Somewhat more than a year ago I listed three compelling reasons to have Do-Not-Track implemented in Adblock Plus:
The specification evolved quite a bit since my initial implementation and merely sending the header is no longer sufficient — client-side detection is required as well, something that Adblock Plus cannot implement properly. Also, the Do-Not-Track implementation has generally grown more complex than initially anticipated, in Chrome it probably had some performance impact as well. So leaving this to browser developers seems to be the right decision.
Adblock Plus 1.3 for Chrome released yesterday no longer sends the Do-Not-Track header. This functionality will soon be removed from Adblock Plus for Firefox as well.
]]>