Adblock Plus and (a little) more
Measuring the memory use of an SDK (Jetpack) based add-on · 2012-01-04 11:37 by Wladimir Palant
Add-on SDK 1.2.1 added a nifty feature: starting with Firefox 9 the memory usage of add-ons built with the SDK is visible in about:memory. However, if you actually try to use this feature you get lost in the sheer amount of compartments. Each module gets its own compartment and the SDK has lots and lots of them. This even prompted a user to report “zombie compartments” caused by my only SDK-based add-on so far.
While there is nothing wrong with these compartments, getting the total amount of memory used by the add-on is everything but trivial. If you don’t feel like adding together a few dozen values you can also go to about:config (or any other privileged page), press Shift+F4 to open Scratchpad and paste the following code:
let prefix = "resource://AddonIdHere-";
let mgr = Components.classes["@mozilla.org/memory-reporter-manager;1"]
.getService(Components.interfaces.nsIMemoryReporterManager);
let e = mgr.enumerateMultiReporters();
let n = 0;
while (e.hasMoreElements())
{
let reporter = e.getNext()
.QueryInterface(Components.interfaces.nsIMemoryMultiReporter);
reporter.collectReports(function(process, path, kind, units, amount, description)
{
if (path.indexOf(prefix.replace(/\//g, "\\")) >= 0)
n += amount;
}, null);
}
alert("Bytes used: " + n);
Change the value for the prefix variable in the first line to the prefix that is common to all compartments (it depends on the add-on ID, e.g. something like resource://jid0-abcdabcdabcd-at-jetpack if Add-on Builder was used) and press Ctrl+R to run the code. The code will add together all matching memory values and show you the result. For my add-on I got 25 MB on Firefox 9 — quite a proud price tag for 40 lines of code. However, on a Firefox 12 nightly the same add-on takes less than 5 MB so maybe things will improve in future (or maybe it simply depends on how the browser is used, e.g. the nightly build had far fewer tabs open). Note: This probably doesn’t count content scripts, I would guess that their memory use is added to the compartment of the page that they are attached to.
How does this fine-grained memory reporting work, could classic add-ons use it as well? The Add-on SDK uses sandboxes for each of its modules, and Gecko creates separate compartments for such sandboxes. To get this compartment reported properly one only needs to specify sandboxName parameter when creating the sandbox.
Now would it make sense to migrate a large add-on like Adblock Plus to a similar module system instead of the regular JavaScript modules? Or maybe even implement similar treatment for JavaScript modules where each module would get its own compartment? I’m no expert on this but it occurred to me that Adblock Plus regularly exchanges objects across module boundaries. And from what I understood, moving an object from one compartment to another is a non-trivial operation. So while having each module in its own compartment would be nice for memory reporting, it would probably also be a significant performance drawback. I hope that somebody who knows this area better will correct me if I’m wrong.
Comment [7]
Commenting is closed for this article.
Zlip · 2012-01-04 16:09 · #
Is this relevant?, it landed in today’s Nightly x86 update.
https://bugzilla.mozilla.org/show_bug.cgi?id=677294
Reply from Wladimir Palant:
Yes, I think that it is relevant. If I understand it correctly, this change allows creating multiple sandboxes in one compartment. That would mean that one can have a single compartment for the entire extension and load all modules into it – quite an improvement. No more countless entries in about:memory and no more overhead from inter-compartment communication. Too bad that we have to wait 4 months for it. And too bad that this is restricted to sandboxes – getting to that sandbox from UI code or a different extension will be non-trivial, nothing like accessing JavaScript modules.
Anonymous · 2012-01-04 16:50 · #
>Name: Wladimir Palant
>Street: Venloer Strasse 151-153
>City: Koeln
>Postal Code: 50672
>RCountry: DE
>Phone: +49.22165028598
Did you really think you would get away with this? By the end of this week I’m going to go make an example out of you. Sellouts will be dealt with as long as I’m around. I’m on the side of justice.
The three things I hate the most are 1) when people make deals in smoked filled rooms, 2) when people censor comments and 3) when people sell their souls for power.
You’re guilty of all three. That is why you have to go. So long Wladimir Palant, you’re no longer relevant.
pd · 2012-01-04 17:31 · #
That code is relatively simple and short and can be run from unprivileged space. Why on earth then hasn’t something like this been already incorporated into about:memory?
Perhaps Wladamir you could contribute it yourself? Seems like nobody else – despite the 6-month-long MemShrink effort – is thoughtful enough to do so.
Jeff Griffiths · 2012-01-04 17:38 · #
The Jetpack project has a bug for this, aptly named ‘Too many compartments!’:
https://bugzilla.mozilla.org/show_bug.cgi?id=672443
I think we’re making good progress on this underlying aspect of SDK-based Addon memory usage. Another key change recently was the move to a new module loader design in the 1.4 branch ( to be released Jan. 10 ). Wladimir, I’d be curious to see if there is any difference packaging your addon with the current 1.4 codebase instead.
Reply from Wladimir Palant:
At the moment I have 0.5 MB instead of 25 MB – quite an improvement. Still have to see how this changes over time however.
Reply from Wladimir Palant:
Sorry, my bad – wrong ID inserted. It’s actually 10.5 MB now. Still an improvement but not quite as drastic.
pd · 2012-01-04 17:55 · #
Could you give some more instructions on how to use this code? Does Firefox have to be from the SDK prompt? Is there a similar web console way to determine whether installed add-ons were written with the SDK? I could go through all my extensions in my profile manually to look for whichever characteristic that all SDK add-ons share, but is there a better way?
Reply from Wladimir Palant:
I’ve changed the instructions to use Scratchpad instead of Web Console – for people who use neither the Scratchpad should be simpler. Note that the code requires chrome privileges which is why you have to open Scratchpad on a privileged page (about:config is one).
njn · 2012-01-07 12:13 · #
Wladimir, do you have a link to this add-on?
Reply from Wladimir Palant:
https://addons.mozilla.org/addon/google-search-link-fix/
senate · 2012-01-11 01:54 · #
Sorry for commenting under an unrelated post but you turn off the comments…
How come you don’t leave the whitelist creating to others like you do with blacklists? Even if I agreed wholeheartedly with you on whitelisting “acceptable ads”, I strongly dislike the fact that you implemented the list in the addon itself, whereas you don’t even do that for blacklisting! Thus, you should rename your extension to AdAllow Plus, since that is what it does by default (not block but overrides what you have blocked).