Adblock Plus and (a little) more
Binary XPCOM components are dead - js-ctypes is the way to go · 2011-07-12 13:19 by Wladimir Palant
Daniel Glazman is shocked to see how hard shipping binary XPCOM components with an extension became now. Fact is, we simply didn’t notice the hidden message of blog posts announcing dropping binary compatibility (meaning that your component needs to be recompiled for each new Firefox version, no matter how simple it is) and rapid releases — binary XPCOM components in extensions are deprecated. Theoretically, somebody could still continue using them but it requires so much effort that nobody can be expected to do that. Unfortunately, I haven’t seen it said like that anywhere, hence this blog post. There is still tons of documentation on binary XPCOM components on MDN and no deprecation warnings. Even XPCOM changes in Gecko 2.0 page lists all the important changes without making any conclusions.
In reality, as an author of an extension that relies on binary code you should start looking at js-ctypes. If the point of a binary component was simply calling some platform functions — js-ctypes can do that for you. If you require some native code (e.g. for cryptography functions that would be too slow in JavaScript) — you can move it into a regular native library and ship that library with your extension. If you do that, don’t forget adding <em:unpack>true</em:unpack> to your extension’s install manifest, the library can only be loaded if it is unpacked into a file on disk. Use AddonManager.getAddonByID() and then Addon.getResourceURI() to locate the library on disk and open it with js-ctypes.
Update: I am closing comments because I won’t be able to follow the discussion. Jorge wrote a pretty thorough reply and you might just as well reply there.
Comment [11]
Commenting is closed for this article.
Daniel Glazman · 2011-07-12 13:59 · #
Ok for thirs-party binary libs even if js-ctypes can be really painful, in particular for multi-platform add-ons having one 2rd-party underlying lib per platform. But what about “native” xpcom components entirely built based on Moz SDK? Do you also think they’re on path to extinction?!? Every now and then, my components stop building. It’s already a huge pain, changes are rarely documented, the xpcom examples are not up to date, but these components are still extremely useful…
Reply from Wladimir Palant:
Yes, they are definitely on path to extinction – Moz SDK is increasingly becoming something for application builders only (meaning mainly Mozilla itself), not for extension developers. It doesn’t matter if you restrict yourself only to the most basic and stable interfaces, recompiling for each new Firefox release is now a feature of the platform. I think that Mozilla has given up on binary XPCOM interfaces for anything other than connecting libraries within Firefox itself. They are streamlining everything to stay competitive in the changed browser landscape, and binary XPCOM components are far more trouble than they are worth. This is only one part of an overall tendency, it looks like “the Mozilla platform” is generally going away to be replaced by “the Firefox application”. Maybe I’m just pessimistic as you say but I don’t like the current course at all (and it isn’t because of the rapid releases).
pd · 2011-07-12 14:32 · #
Asa is getting paid to be community liaison right? There are many stakeholders that can be considered under the label ‘community’ including Add-On developers using binary components. If these valued contributors to the ‘community’ are not being informed of key changes to the Firefox codebase that will effect them, is Asa failing to do his job? Add that to his enormous ‘enterprise’ gaffe and do we get a good case for firing him?
Reply from Wladimir Palant:
Add-on developers aren’t Asa’s job, Asa lacks the knowledge for that. In fact, informing add-on developers is a task spread out between core developers, MDN and AMO. There is no single person with the job description being “think about implications of our changes for add-ons developers and communicate them” and unfortunately, we had several important changes lately where nobody did think them through properly. Add-on developers themselves tend to notice the issues far too late, when a release is getting close and code stability requirements prevent any significant changes.
Daniel Glazman · 2011-07-12 14:42 · #
@Wladimir: then I think this is something the ecosystem will react to, once again. I don’t want to be pessimistic, I want to be realistic.
Reply from Wladimir Palant:
The ecosystem is already reacting. Application developers started droping XULRunner in favor of other solutions a long time ago. Add-on developers are increasingly looking over to Chrome. I sincerely hope that I will never see the day when MoCo is all the ecosystem there is but that’s where we are heading.
Someone · 2011-07-12 14:49 · #
@pd: maybe anyone could file a “Replace Asa” enhancement request on BMO? ;)
pd · 2011-07-12 14:58 · #
That is unacceptable.
Maybe it’s time a developer took the chief lizard wrangler role instead of an acrobat? That might lead to oversights like this being addressed?
Reply from Wladimir Palant:
You are being unfair to Mitchell Baker – she has been doing a great job and organizational questions like this one aren’t her responsibility. Anyway, if you bother to open the Wikipedia page on her you will notice that she is a lawyer, regardless of her hobbies, and that she no longer holds a post within MoCo.
Some people did spend some thoughts on how to organize communication with the add-on developers given the rapid release process. The current proposal is that core developers report any changes that might affect extensions. AMO then feeds that information into the add-on validator and alerts add-on authors that might be affected. Whether this approach works better (or at all) is still unknown at this point. At least I haven’t seen anybody comment on the results after Firefox 5 release – maybe we will get more info once Firefox 6 is released.
Matthew Gertner · 2011-07-12 15:12 · #
@Wladimir I’m curious to know why you say “I don’t like the current course at all.” I have a feeling I agree with you but I’d like to know more about your reasons.
Reply from Wladimir Palant:
Firefox has serious competition now, that’s an entirely different game than its first five years. That might be a good thing for the web but it made MoCo even more focused on Firefox and market share than it always was. I see lots of pressure being built up – Mozilla needs to deliver fast, even if sacrifices have to be made. Rather than investing into the platform or the ecosystem Mozilla is increasingly investing into short-term goals. One is regularly being reminded that Mozilla’s “customers” are end-users and not add-on developers. There were already signs of this during Firefox 4 development (e.g. the toolbar icon disaster that was luckily averted). The “slow add-ons initiative” was another important sign, a masterpiece of miscommunication and negligence. And there are many smaller signs that it is now ok to screw third-parties if that allows Firefox to move faster. No, I don’t really care whether companies will continue using Firefox – but I am very concerned if they get “piss off” as an answer.
Matthew Gertner · 2011-07-12 16:09 · #
Not to be too cynical but this is hardly anything new. Mozilla announced in May 2007 that they were essentially abandoning the platform (XULRunner). To be fair, it’s a very difficult line to tread since producing a high-quality platform will necessarily take away from the end-user perception of the browser. Also, I understand why they are frustrated by perceptions that Firefox is slower and/or more of a memory hog than Chrome to the extent that this is due to misbehaving add-ons and not the core platform.
Nevertheless I think Firefox add-on ecosystem is its only long-term competitive advantage in the newly challenging browser market you rightly point to. It’s a very complex issue and I need to think and blog about it as well.
Michael Kaply · 2011-07-12 16:15 · #
Last year I was involved in a conversation with some AMO folks and Mozilla developers where it seemed like they understood the need for some central person to be a go between between Firefox development and add-on developers. Then they dropped the ball.
What is needed (was needed?) is a person who is very familiar with Firefox code and addons. Most Firefox developers have no clue that what they do even affects addons (hence the icon fiasco).
I’m becoming very pessimistic as well though. I believe the end goal here is to drive traditional add-on developers out of business and move everyone over to Jetpack.
This quote from fligtar is priceless:
“You can do nearly everything with an Add-on SDK and Add-on Builder based add-on that you can do with a XUL-based add-on,” said Justin Scott, Mozilla’s add-ons product manager.*
I guess he’s never built an interesting add-on.
*Read more: http://news.cnet.com/8301-30685_3-20074590-264/rapid-release-firefox-meets-corporate-backlash/
Reply from Wladimir Palant:
Yes, allowing very powerful add-ons is probably the most significant advantage that Firefox has over Chrome. I don’t think that anybody would be stupid enough to pursue dumbing down the Firefox extension system to the sad level where Chrome is right now. Then again, some statements make it look like Mozilla plans to neglect “traditional” add-ons – to the point where supporting them becomes too much effort. I sincerely hope that this is a wrong impression.
Boris · 2011-07-12 18:33 · #
Wladimir, we’re heavily investing into the platform. The problem is that investment in the platform and “don’t change anything” are mutually contradictory, and a lot of people seem to be looking for the latter….
Now there’s the separate issue of how easy the platform is to build on if what you’re building is not a webapp. In some ways, what’s going on is that building webapps is becoming simpler and they’re becoming more powerful. To the extent that this happens and that webapps like that can replace XUL addons, it’s OK to deprecate XUL addons slowly. Of course that means deprecating the less powerful ones that weren’t hooked deep into the system anyway.
Justin Scott · 2011-07-12 19:30 · #
While we haven’t done a dedicated blog post on the topic because of the very few add-ons on AMO that use binary components, our posts on the the topic of compatibility have mentioned that binary components need to be recompiled with every version: http://blog.mozilla.com/addons?s=binary
Additionally, in the compatibility bump email notifications, any binary add-ons are told that we could not bump them because they must recompile binary components with every release.
Second only to non-hosted add-ons, binary add-ons are the biggest compatibility problem with moving to rapid releases. It’s very unfortunate that they have to be recompiled and we are encouraging authors to move to js-ctypes when they can. Kev has been reaching out to all of the most popular binary add-on companies (such as toolbars and antivirus add-ons) for many months warning of what’s to come and encouraging a move to js-ctypes.
@mkaply While that quote is attributed to me, it actually came from https://wiki.mozilla.org/Labs/Jetpack/FAQ
Jorge · 2011-07-12 19:48 · #
Well, communicating breaking changes to the add-on developer community is a very well-defined part of my job.
Learning about those breaking changes from the development team is where it gets complicated. I have to go over bug lists every 6 weeks, trying to identify those that affect us. It has been pretty effective in general, but it’s very dependent on bugs having descriptive summaries.
It’s also difficult for Firefox developers to assess which bugs will affect us or not. Pretty much any (IDL) interface or UI change can potentially do this, so I can understand why this isn’t on every developer’s mind at the moment. It’s something we’re working on.