Adblock Plus and (a little) more
Why I had to ban China - fighting spam · 2010-07-26 15:54 by Wladimir Palant
I want to start with an apology to the few Chinese who are using this website for legitimate purposes — next time they try to post on the forum or comment on my blog they will most likely see an error page. Which is really a pity and I feel bad about that. But I cannot really see another way.
Most people probably know already, spam on the web isn’t limited to e-mail. Spammers discovered forums and blogs as a new platform for spam and are increasingly targeting them. And the more popular a website is (in terms of Google rank), the more spam it will get.
One has to realize that this is no longer about spam bots — these can be kept out with relatively simple technological solutions. Unfortunately, there is a tendency to hire people for “data entry jobs”, particularly in countries like China or India. These people will send spam posts to forums and blogs all day as their job and solutions targeted at defense against bots will help very little. And they will try to disguise their posts as real questions as much as possible, sometimes even resulting in other people wasting time on trying to help them.
For a long time the solution was to have moderators remove spam in forums, I would then occasionally ban IP ranges to keep out persistent spammers. My blog was generally less targeted and I could clean out the spam myself. Unfortunately, the amount of spam has been building up over time (probably also because of this site becoming more popular). And now it got so bad that I had to keep fighting spam while on vacation — spam comments on my blog almost daily, often with many different blog posts spammed. Similar picture in the forum where some spammers spend hours (not an overstatement) to spam dozens of topics.
With the bulk of spam coming from China I saw no other way but to ban Chinese IP addresses altogether — these users will still be able to read or download everything but posting to the forum or to this blog will no longer be allowed. As I said, this is not a decision I am happy with but it will hopefully make moderator’s life a lot easier. I checked the existing forum posts against the IP database and it should be very few users who will be affected by that measure.
Comment [5]
Commenting is closed for this article.
Nathaniel Kofalt · 2010-07-26 17:11 · #
I completely sympathize. Running a small Apache instance for an audience of just a few dozen (no high PageRank or similar), yet I was dismayed that illegitimate spam, vulnerability scans, and general attacks took up more bandwidth/log space/CPU than legitimate traffic. I ended up doing the same thing, though I forget where I got the list from.
Which service did you use?
I just discovered blockacountry but there are dozens of lists out there.
Reply from Wladimir Palant:
I simply pulled the IP ranges from MaxMind’s GeoLite Country database (http://www.maxmind.com/app/geolitecountry). I already use that database on the server for log processing and it is updated monthly.
charlo · 2010-08-02 23:57 · #
Sympathizing too. The only solution is very extreme and saddens me. Human robots, then. Just what can one do against it ? human moderators, that is, but so time consuming…
Well, I would not be too sad to not be able to post here (with my english so perfect I can not express what I would like to say), but it makes me think about the ability to “participate” on the web in general.
Miron · 2010-08-07 17:58 · #
For instance a Chinese User will able to post one message in 24 hours.
The users should go through validation procedure before posting (Prove you are human and know whats Ad-block).
After few weeks this user will be cleared.
This is a bit complicated but give it a try.
Reply from Wladimir Palant:
While some Chinese spammers will keep posting new messages, for most one spam message in 24 hours is all they need – they usually won’t come back to the same forum for weeks anyway. This approach will not help much unfortunately.
As to validation, we already validate that the user is human. But how do you want to validate that he knows about Adblock Plus? What question do you want to ask that cannot be typed into Google? Mind you, these guys dig up old questions and reformulate them slightly just to make their spam posts look valid, a simple question won’t stop them. And a complicated question will stop most people having real questions.
devskii · 2010-08-13 10:04 · #
i remember when captchas were very effective in fighting spams and auto registrations, until the spammers use human captcha typist lol. others use third party comment system like disqus but got more problems ..
Christophe · 2010-08-20 16:28 · #
How about using OpenID? Not requiring authentication is the best way to invite blog comment spam.
Reply from Wladimir Palant:
Authentication is only somewhat useful against bots. The human spammers pretty much always register before posting – even though I don’t require it. So I doubt that OpenID will stop them. It will stop many of the real users however.