Adblock Plus and (a little) more
Expected server downtime · 2010-11-05 14:02 by Wladimir Palant
The server will be unreachable for some time later today. This is necessary to move adblockplus.org to a more powerful server which should hopefully resolve the connectivity issues lately. Everything should be done within 30 minutes. However, it might happen that the IP address of the server changes — in this case it will take a while until everybody can reach the server again (due to DNS caching).
Update: It looks like nothing will be happening today. I guess that process isn’t as automated as I though. Probably on Monday then?
Update2 (2010-11-08): The move just happened, took only 10 minutes. I upped the limits and hopefully the “currently unreachable” message will no longer show up.
Comment [3]
Commenting is closed for this article.
A GREATFUL USER · 2010-11-05 18:15 · #
NICE JOB! LOVE THIS ADD-ON! BY FAR MY FAVORITE SO FAR. KEEP UP THE GREAT WORK – THANKS!
cider · 2010-11-08 00:35 · #
Can you publish some info about the webserver? I am very interested in web site performance and general info on high traffic webserver setups.
I see that both adblockplus.org and easylist-downloads.adblockplus.org share the same server at 1and1 which runs nginx as a webserver. I guess you use the HttpGzipStaticModule to send out the filter subscriptions pre-zipped, right? Do you even use kzip or 7zip to get smaller files?
How much traffic do you have on that machine? Which CPU and how much RAM do you have? Whats the bottleneck? Don’t you max out the 100MBit NIC? How much concurrent connections? Whats the load of the server? How much traffic goes to ares2.org to download the lists?
Maybe you could do a Blogpost about the tech behind the lists distribution.
Best,
cider
Reply from Wladimir Palant:
It’s nothing spectacular. This server isn’t handling EasyList downloads alone – there is some client-side load balancing (in the Adblock Plus extension) and the main portion of the traffic is being handled by ares2.org. We can add mozdev.org as a mirror if things get tough (it doesn’t support HTTPS so we prefer not to) and there are plans to add more mirrors in future. So adblockplus.org isn’t even a dedicated server, currently handling about 600 GB traffic per week.
You guessed correctly – it is nginx with HttpGzipStaticModule, and I use 7zip for better compression. Nginx is perfect for this kind of load, both CPU load and memory use are absolutely negligible. We did have a spike in CPU load shortly after switching to HTTPS but that’s gone now. I guess that newer nginx versions improved their HTTPS handling. So now the problem is only that connections stay open longer, consequently we have a larger number of concurrent connections. This is why the move was necessary, to get a higher connections limit.
cider · 2010-11-08 17:04 · #
thanks!
Best,
cider