Adblock Plus and (a little) more

Translating Adblock Plus: Dumping Babelzilla for Crowdin · 2012-11-09 13:40 by Wladimir Palant

The Adblock Plus project has been using Babelzilla for translations for more than six years. Yet, time has come to change that. Two days ago I moved the translations for all my extensions from Babelzilla to Crowdin, meaning Adblock Plus, Element Hiding Helper and JavaScript Deobfuscator. This was preceded by a lengthy pilot phase with the Customizations for Adblock Plus extension and recently also Adblock Plus for Chrome.

Why change to a new platform?

Babelzilla helped us a lot, managing more than 40 translations for Adblock Plus alone wouldn’t have been possible without it. Yet, I’ve been dissatisfied with the service for quite some time (mentioned it first time two years ago). Reasons are:

• Bugs. Yes, bugs are unavoidable, each project has them. However, some projects manage to fix them and do so without regularly introducing new bugs. Unfortunately, each translation round lately hit new bugs, occasionally corrupting the translations to the point that they had to be fixed manually.
• User interface. I felt bad each time I sent people willing to contribute to Babelzilla. It was very obvious that 90% of the time they wouldn’t figure out how to use it. Locale code ambiguity made things significantly worse: e.g. Babelzilla allowed an Italian locale to use both it and it-IT as the locale code, a “feature” that caused lots of confusion among translators with duplicate translations created all the time.
• Collaboration. The current translation process doesn’t encourage the creation of stable translation teams, favoring one-man translations instead. Worse yet, it makes it hard for somebody else to continue the work once that translator leaves. As a result we have a number of abandoned translations and several that are known to be of low quality.
• Communication. The dedicated communication channel (a forum topic) was apparently something that many developers simply didn’t know about. So I had to send out private messages all the time, a rather time-consuming task.

What about alternatives?

I’ve looked at lots of different options to manage our translations. There was a thought to use Anwiki but it has lots of issues of its own. Pootle was a consideration but it didn’t look like a great solution (e.g. very much geared towards .po files) and we would still have to host it ourselves.

For a while it looked like Tim Babych’s Adofex would be the way to go, especially after Babelzilla announced plans to move to that platform. So early this year I approached Babelzilla admins with some thoughts on how to make that migration progress faster and generally make Babelzilla a more reliable platform. After not hearing back I finally realized that depending on somebody’s hobby project with something as important as localization might not have been the best idea.

What does Crowdin have to offer?

I came across Crowdin in July thanks to a hint from Thomas Greiner. I decided to try translating the Customizations extension there, one that wasn’t on Babelzilla simply because I was afraid of uploading it there due to all the bugs. Here is my experience so far:

• It has bugs and quite a few things could be improved, no doubt about that. So there is a reason why it says “beta” in the logo. However, the support has been very responsive and the serious issues have been resolved very fast. I can clearly see the tendency towards better quality.
• It’s a global translators community very much like Babelzilla and we actually gained a few new translators in the transition, something that wouldn’t have been possible with a private translation solution.
• Collaboration of multiple translators is easy: anybody can suggest translations or vote on the existing translations. And it is possible to appoint proofreaders (experienced translators) who will approve one of the proposals. Plus there is a discussion group where translators for a particular language can talk.
• It is possible to help translators by providing more information – either by using screenshots (any number of strings can be assigned to a screenshot) or by creating a glossary with important terms and explanations. Requires quite a bit of effort however, I wasn’t able to do that across all translation projects so far.
• There is also automatic translation aid like translation memory and machine translations.
• Translators can ask questions about a particular string in the comments section, they can also notify the project owner about problems by checking “Create issue” checkbox.
• The API is great! It’s very easy to automate all the usual operations like uploading new strings or getting the translations back. And they are working on a Ruby library to use that API as well as a command line tool.

Bugs? What bugs?

Crowdin’s support for the DTD and properties file formats isn’t ideal, they manage to extract the strings from these files all right but not the comments. This is not really surprising given that the way how comments in these formats should be matched to strings is “underspecified”. Fortunately, Crowdin supports a number of other file formats as well. So my solution was to convert all locale files to the Chrome-style JSON format on upload and back on download. This format has the advantage of a well-specified description field and it is generally a lot easier to parse due to the availability of good JSON parsers for basically any programming language.

Another major issue are access keys. Crowdin doesn’t always display the strings in the order in which they are listed in the file, in particular untranslated strings are always listed first. This has the side-effect that locating the label for an access key is sometimes non-trivial. Which is of course a side-effect of translating labels and their access keys as separate strings — not a good idea in the first place. I consider merging them into one string for the translations in future.

There are also some minor annoyances like there being a limited number of supported languages (I requested adding all the additional languages supported by Firefox but that appears to be non-trivial). Or the not quite yet mature placeholders support. Or the overly crowded (pun intended) translation interface.

Tags: adblock plus mozilla

Comment [3]

1. Goofy · 2012-11-09 17:14 · #

   We are happy you finally found a better toy to play with. Good luck, Wladimir.

2. Eric Jung · 2012-11-12 19:02 · #

   Hi Wlad,

   Thanks for the post. Lots of the top 25 addons have been looking for alternatives. I expect crowdin will get a lot of new business because of your choice.

   So my solution was to convert all locale files to the Chrome-style JSON format on upload and back on download

   Is that open-source? If not, are you willing to share it? FoxyProxy is going to try crowdin. It would be nice to follow in your footsteps re: JSON conversion. We use ant (build.xml) and jenkins for our build system, so JSON conversion and upload/download via API would be a great fit.

   Thanks,
   Eric Jung

   Reply from Wladimir Palant:

   Yes, it is open source, part of our build system in fact (https://hg.adblockplus.org/buildtools/file/024d54387a9d/localeTools.py). Not sure how much you can reuse that but the important functions are parseDTDString and parsePropertiesString as well as toJSON/fromJSON.

3. Crowdin Team · 2012-11-13 17:47 · #

   We will be happy to provide you solution that will not require any additional tools to manage your localization files. Can you please ping us at support@crowdin.net and let us know the bugs you experience with DTD files?

   We’re online and happy to help.

   //Crowdin Team

Commenting is closed for this article.