Adblock Plus and (a little) more

Modularization in a restartless extension · 2012-07-12 17:21 by Wladimir Palant

A simple restartless extension can probably keep all its code in the bootstrap.js file. However, it gets crowded there very soon. Plus there is some code that is really only boilerplate and should probably kept separate from your actual code.

This sounds like a job for JavaScript code modules. It is mostly a matter of taste (I prefer CommonJS module syntax) but there is one really big disadvantage of JavaScript code modules: they have to be unloaded explicitly when your extension is shut down. Which means that you either have to keep a list of modules to unload in your bootstrap.js file or add cleanup code each time you load a module. I find neither approach very compelling.

Custom module loading

Fortunately, a custom module loader is simple. This is the most trivial implementation for the CommonJS require() function:

Components.utils.import("resource://gre/modules/Services.jsm");
var addonData = null;  // Set this in startup()
var scopes = {};
function require(module)
{
  if (!(module in scopes))
  {
    let url = addonData.resourceURI.spec + "lib/" + module + ".js";
    scopes[module] = {
      require: require,
      exports: {}
    };
    Services.scriptloader.loadSubScript(url, scopes[module]);
  }
  return scopes[module].exports;
}

This function keeps the scope for each module in a variable. If the scope isn’t there yet (module not loaded yet) then it will create a new scope, add itself and exports as global variables into that scope and load the module file into it. The module URL is being constructed using the information that the extension gets in its startup() function (here the modules are expected to be placed in the lib/ subdirectory of the extension). And when the extension is disabled or uninstalled bootstrap.js will be unloaded and all the module scopes referenced here will be garbage collected automatically.

There is one important detail to note: the module scope in the example above is a plain object. This means however that the global variables from bootstrap.js will be accessible to the module, something that is often undesirable. You can prevent this by using a sandbox instead of an object:

let principal = Components.classes["@mozilla.org/systemprincipal;1"]
                          .getService(Components.interfaces.nsIPrincipal);
scopes[module] = Components.utils.Sandbox(systemPrincipal, {
  sandboxName: url,
  sandboxPrototype: {
    require: require,
    exports: {}
  },
  wantXrays: false
});

This will give the module an isolated scope — but also an own memory compartment. Mozilla developers work hard on making compartments and communication across compartments cheap but at the moment having multiple compartments in your extension can still increase the memory use significantly (at least that’s what I saw with Adblock Plus). Also, you will no longer be able to check the memory use of the extension on about:memory easily: instead of looking up one compartment you will have to add up multiple. So each approach has its advantages and disadvantages.

Injecting additional variables into module scopes

One advantage of a custom module loader is that you can predefine more variables than just require and exports — you can add anything to the scopes that you like. So I add the variables Cc and Ci to all modules, otherwise all of them would have to define these variables explicitly:

scopes[module] = {
  require: require,
  exports: {},
  Cc: Components.classes,
  Ci: Components.interfaces
};

Accessing modules from user interface code

All your modules get the require() function injected into their scope automatically. But what if you have some code that isn’t a module? For example, your extension opens a dialog window and it has to access one of the modules. That’s where a custom module loader has a disadvantage compared to the built-in code modules. My solution was using an observer message that could be sent whenever some external code needed access to a module. So bootstrap.js would have the following code:

Components.utils.import("resource://gre/modules/XPCOMUtils.jsm");

var RequireObserver =
{
  observe: function(subject, topic, data)
  {
    if (topic == "myextension-require")
    {
      subject.wrappedJSObject.exports = require(data);
    }
  },

  QueryInterface: XPCOMUtils.generateQI([
    "nsISupportsWeakReference",
    "nsIObserver"
  ])
};

That observer needs to be registered in the startup() function:

Components.utils.import("resource://gre/modules/Services.jsm");
Services.obs.addObserver(RequireObserver, "myextension-require", true);

And removed in the shutdown() function:

Services.obs.removeObserver(RequireObserver, "myextension-require");

This observer makes sure that user interface code can use the following require() function which has the same effect as the require() function in a module:

Components.utils.import("resource://gre/modules/Services.jsm");
function require(module)
{
  let result = {};
  result.wrappedJSObject = result;
  Services.obs.notifyObservers(result, "myextension-require", module);
  return result.exports;
}

Dealing with memory leaks

I mentioned above that module scopes will be garbage collected automatically when the extension is shut down. This assumes however that there are no external references left. For example, if your module foo has a function bar() that you register as an event listener in the browser window and forget to remove when the extension is shut down — this will prevent function bar() from being garbage collected. The function in turn will prevent the module scope from being garbage collected, and the module scope holds a reference to the require() function which holds a reference to the bootstrap.js scope which holds a reference to the scopes variable. Oops, a single leftover event listener prevents your entire extension from being garbage collected.

The good news is that you will notice that: your extension is disabled but going to about:memory?verbose still shows a compartment belonging to it even after minimizing memory use. This means that there is a memory leak and you can hunt it down. However, in one case I simply couldn’t do anything about the leak: the only way to remove the external reference was for the user to reload all his web pages. Is it really unavoidable to leak everything in this scenario?

In fact, it isn’t. You can avoid it by destroying the references between modules. You have their scopes, if you remove all global variables from them it should usually be sufficient to prevent everything other than that one module from leaking. You can run the following code when shutdown() is called:

for (let key in scopes)
{
  let scope = scopes[key];
  for (let v in scope)
    scope[v] = null;
}
scopes = null;

Now you still have a memory leak but at least its impact is limited.

Tags: gecko

Comment [6]

Jeff Griffiths · 2012-07-12 18:09 · #

Not sure if you are aware of this, but the loader implementation from Jetpack has landed on Mozilla Central and will be available in Firefox 16. Eventually we will land the core apis as well, but one goal is to allow add-on devs ( and Firefox devs! ) to write and use CommonJS modules using this loader.

Reply from Wladimir Palant:

No, I didn’t know that something landed already. Do you have a link?
Jeff Griffiths · 2012-07-12 20:06 · #

Here’s the commit:

https://hg.mozilla.org/mozilla-central/rev/efa8bb276e3d

Reply from Wladimir Palant:

I see – not in the build yet. From the look of it, this loader seems to come with a significant overhead (like most mechanisms in the SDK unfortunately). Regular extensions probably don’t have as much interest in putting their modules into a virtual Alcatraz :)
Tobu · 2012-07-15 15:54 · #

@Jeff Good to see. Instructions about rewriting the module loader felt pretty scary when I’m willing to let a framework like the addon-sdk do it for me.
T · 2012-07-19 18:58 · #

Hi, the ‘download’ button gives me a 404 error. Could you please fix? Thank you!

Reply from Wladimir Palant:

Mozilla is fixing it – the problem is on their side. In the meantime, just restart Firefox and try again.
real estate lawyer Newport Beach · 2012-08-10 14:07 · #

Thanks for the magnificent blog. Your blog about a Adblock Plus and (a little) more and i am sure people like this blog. Where else may anyone get that type of information in such a perfect manner that also involves great writing skills? I have a presentation subsequent week, and I was looking for such information. Thank you for making my search a great success.
real estate lawyer Newport Beach · 2012-08-14 11:36 · #

Incredible blog. You have provided the lot of useful and interesting information in this blog post and Your blog about a Adblock Plus and (a little) more. It’s great to read those amazing tips that are shared here. There’s no doubt that I will ultimately make a move with your tips on those things I could have never been able to handle alone. You are so considerate to allow me to be one of those to learn from your beneficial site. I appreciate your efforts from bottom of my heart.

Commenting is closed for this article.