Overview for Mozilla's add-on performance measurements · 2011-04-11 16:55 by Wladimir Palant
I got tired of digging through logs so I created an overview page for AMO’s add-on performance testing results. This shows both the absolute slowdown values as well as percentages for each operating system, along with a link to the corresponding logs. It also shows the standard deviation for each test and flags results that are affected by issues known so far. This week this means:
- 9 add-ons that weren’t tested because of being platform-dependent (bug 648225)
- 2 add-ons (Google Toolbar and Memory Fox) weren’t tested because they have no stable version for download on addons.mozilla.org
- 1 add-on (Tab Mix Plus) wasn’t tested because Talos couldn’t understand its install.rdf file (bug 648978)
- 4 add-ons where some of the tests timed out, typically because of first-run pages that couldn’t be closed (bug 648222)
- 6 add-ons that were tested disabled because of not being compatible with Firefox (typically Thunderbird add-ons)
- 30 add-ons that were tested disabled because of not being explicitly compatible with Firefox 4 (bug 648229)
- 11 add-ons with test results probably considerably skewed by bug 648734 if I count correctly (standard deviation above 10% of the total Firefox start-up time). A lot more results were marked with a warning because of standard deviation above 2% of the total Firefox start-up time but I won’t count those.
- Almost every add-on has its Windows results skewed by bug 647954. This apparently works in both directions, for some add-ons the Windows score has been increased due to errors, others got an advantage because their initialization didn’t run.
I guess I will have to re-run that script weekly until AMO fixes the bugs and starts providing affected add-on authors with useful information.
Commenting is closed for this article.