Adblock Plus and (a little) more
Some more details on Mozilla's add-on performance measurements · 2011-04-06 14:35 by Wladimir Palant
Mozilla Add-ons recently published an article on their effort to recognize slow add-ons. It comes complete with a list of worst offenders. Some media already picked up on that article and users are listening up. So everything is progressing nicely towards the goal — pressure on developers starts building up, I already had something like 10 people point out to me that Adblock Plus is number 12 on that list.
I can however imagine many developers being not too happy right now, and not only because of more attention to the shortcomings of their add-on. Adblock Plus for example already follows all the Performance Best Practices that apply to it, how comes it is supposedly degrading Firefox startup performance by whooping 21%? What is being measured there, how, under which conditions? How much delay are we talking about, in absolute numbers?
So I got Talos source code from http://hg.mozilla.org/build/talos/ and looked around. I couldn’t find the configuration files used to run add-on tests but I could find the raw test results under http://tinderbox.mozilla.org/showbuilds.cgi?tree=AddonTester (you have to go a few days back, the last round of tests ran on April 2nd). So to sum up the findings:
- Right now there are four machines running the tests, on Windows XP, Windows 7, Mac OS X 10.5.8 and Fedora Linux 12. The application measured is Firefox 4 everywhere. The percentages in the “blame list” are probably the averages for all four platforms (weighting more popular platforms stronger would make sense but this doesn’t seem to be the current approach).
- Talos will always first run Firefox with a newly created profile to do various first-run tasks and initialize caches. Then it will do 20 measured runs with the same profile. Out of these, the first always takes longer for some reason and is ignored as an outlier — only the last 19 runs are counted for the average.
- The regular Firefox start-up time on these machines varies between 400 ms (Windows XP) and 700 ms (OS X). So “20% slower startup” means 100-150 ms delay. Interestingly, on my rather slow Windows 7 laptop the delay caused by Adblock Plus is still 100ms despite a regular Firefox startup taking 800 ms. So either Adblock Plus delay (unlike Firefox startup times) doesn’t depend on hardware or Adblock Plus performs very badly on non-Windows platforms.
- Extensions that require some special activation (like Adblock Plus: add a filter list) have an advantage, this activation currently doesn’t happen. I pointed this out and apparently the solution is to pick out some extensions manually and make sure they are fully initialized during performance testing. Which puts extensions where this doesn’t happen at an even larger advantage.
- Talos will currently uncompress all extensions when installing them, regardless of
<em:unpack>flag. This is not the way Firefox 4 usually works, extensions are left in the compressed XPI file. I tried fixing this — Adblock Plus startup delay increased by roughly 15 ms. Ok, so much about compressing XPI files with the maximal compression factor, for a smaller download. What if my XPI package isn’t compressed at all? Then the startup delay decreases by 15 ms and we have only 85 ms disadvantage compared to vanilla Firefox. Nice but I don’t want to distribute a 1.5 MB download. Turns out that using minimal compression level still has a few milliseconds advantage over an unpacked extension while only increasing the download package by 20 KB — that’s acceptable. Anybody want to update performance best practices? - The current test results are probably significantly skewed on Windows platforms due to a bug in the extension extraction routine.
Here are my own testing results on Windows 7 (rounded, these numbers aren’t very precise). I tested various Adblock Plus setups as well as Firebug for comparison (sorry John Barton).
| Configuration | Time (ms) |
|---|---|
| Vanilla Firefox 4.0 | 800 |
| with Adblock Plus 1.3.6 unpacked | 900 |
| with Adblock Plus 1.3.6 packed at level 9 (current download package) | 915 |
| with Adblock Plus 1.3.6 packed at level 0 | 885 |
| with Adblock Plus 1.3.6 packed at level 1 | 895 |
| with Adblock Plus 1.3.6 and EasyList | 1160 |
| with Firebug 1.7.0 unpacked | 1350 |
| with Firebug 1.7.0 packed | 1310 |
As the test results for Adblock Plus with EasyList (here tested with all the caching introduced in Adblock Plus 1.3.5) indicate, we have bigger worries than whether Adblock Plus is installed packed or unpacked. Having a filter lists adds 250 ms to the startup time for me. Out of these, 200 ms are spent by Gecko parsing the element hiding stylesheet — not something I can really influence. The only way is having EasyList reduce the number of filters somehow (currently it has 11 thousands).
Comment [1]
Commenting is closed for this article.
johnjbarton · 2011-04-07 19:05 · #
Thanks for adding Firebug to your table Wladimir. It’s great to see some real numbers rather than %.
I guess that bug 639898 can’t really be involved in the startup testing. That would activate Firebug for all pages on startup, that would be quite a bad startup experience.
One thing I’m curious about: what does startup time mean? I guess for both Adblock and Firebug the addon has to be looking at the initial Web page loads or the user is not getting what they asked for. If startup time means from the time the user launches Firefox until they are reading a web page, then how can any lazy loading make a difference?
Reply from Wladimir Palant:
Yes, I forgot to write about that in my blog post – the startup time is measured as the time until a page loads. That’s a file:/// page that will report the time of the “load” event. Which allows for some optimizing on my side. Not really worth it however – other than Talos nobody uses file:/// as their homepage. Real users will be increasingly switching to about:home which is perfect for ABP’s delayed initialization.
Reply from Wladimir Palant:
Btw, the actual measurement results for Firebug from April 2nd are 920, 738, 1141 and 901 milliseconds on Windows 7, Windows XP, OS X and Fedora. The corresponding reference times are 542, 401, 700, 496 milliseconds.