Blog spam: where is this gonna end?
Filed under: cyberspace
Number of comments: 
Number of trackbacks: 
Tagged with: Today I got a mail from Dreamhost telling me they had to temporarily shut down this website because it was eating up way too much CPU time. I wondered why that could be because it had been running fine for months already and I hadn't made any recent changes that could result in an unusual CPU load. A quick tail -f on my http access log told me what was causing it.
Some pathetic moron with a botnet was totally HAMMERING my site with POST requests. As a matter of fact, at the moment I'm writing this post, he still is. This particular asshole was quite easy to detect and stop because I found he's flooding my site with POST requests on regular PHP pages. Of course those aren't supposed to be POSTed to at all which is why I could quickly bring my CPU usage back to an acceptable level by adding the following to all relevant PHP files:
if($_SERVER['REQUEST_METHOD'] == 'POST'){
die();
}
Note that I wish this whole die(); thing could be implemented literally this time. What a bloody moron. What a waste of time and resources. This whole thing made me wonder where this travesty that comment spamming is will end. This particular spammer already caused my site to be taken down. This time there was an easy way to get rid of most of the load. But what if there isn't anymore?
Not just us small town bloggers
This morning I read on Elliott Back's blog that even Amazon.com is being seriously hit by spammers. Surfing on for a bit I found that Reddit has serious issues with it as well. And del.icio.us too. This whole thing makes me cringe. It makes me wonder about the future of the internet.
A while ago when I was still actively developing Pivot Blacklist (now deprecated because all important features are in Pivot itself) I received reports from people who had to pay serious amounts of money to their providers due to excessive bandwidth usage caused by comment spammers and referrer spammers. Excessive bandwidth is wasted on my side as well. Luckily I've got 1.1TB to burn at Dreamhost but not everyone has this much bandwidth on his hosting package. Then there's the CPU usage. While I have a lot of bandwidth, I have to make sure I'm not using too many CPU cycles, like anyone else on a shared server. This morning the server load was about 16 when I looked which is why I can't really blame them for temporarily shutting down my site. I got back quite quickly after applying the fix I described earlier but one day, I might not.
Waste, waste and more waste
The amount of bandwidth, CPU time and man-hours to fix things on a global scale must be staggering high and continuously going up. All of this because of a bunch of fuckers (excuse my French) who feel the desperate need to promote their pathetic (often fake) prescription medicines, shady gambling sites or disgusting porn sites. Sometimes I really feel the end of the free, unregulated internet is near. It just can't go on. A lot of companies and people are paying LOTS of dollars for a massive amount of wasted resources. And I'm still only talking blog spam here. I'm not even mentioning the staggering amount of email spamming going on daily.
Mark my words. We'll probably have to enjoy the internet the way it is now while it lasts because if all of this continues it just can't last forever. We might very well be moving towards a totally regulated internet on which providers, owners of backbones and governments decide which traffic can pass and which traffic can't. We'll be dealing with 'preferred sites' and 'preferred networks' which might very well affect all of us. I must honestly admit that I myself have even considered blocking entire countries from certain servers because 99% of the traffic coming from them was abusive. Why? Because the crap just HAS to stop. I didn't end up blocking any countries because I feel it's not the way. It just isn't fair. Not even if only a small amount of legitimate visitors from those countries like to read my site. They should be able to. Of course they should.
Now what?
So far I haven't really seen any REAL solutions. Sure there's Akismet, there's the stuff I created for Pivot, there's tons of other anti-spam tools. None of them however stop the massive amount of wasted bandwidth and CPU resources. They prevent spam from appearing on our sites mighty fine but that just isn't enough. The amount of attacks (yes, today's spam runs closely resemble DDOS attacks) seems to be going up instead of down. 2 years ago I hardly received any attempts to spam any of my sites. Today the requests are slamming on whatever I put online, usually with multiple attempts per minute. 24/7, 365 days a year.
So what do YOU think? Is spam going to kill the internet as we know it eventually or is someone going to come up with a holy grail that will free the internet of this enormously heavy burden? I'm interested in your thoughts!
At 24 June '06 - 16:09 Jeriko One wrote:
One thing I thought about was replacing the traditional submit button of a form with a regular link and don’t apply any action to the form itself. I guess spambots are looking for that in particular, so this might be a solution, although only temporary – unfortunately, this would break validation and I even don’t know, if it’s possible…
At 24 June '06 - 16:13 Marco wrote:
The problem is not at all keeping the spam away from our blogs. That’s the easy part. In fact, we won that fight. I hardly ever get spam. However, their ATTEMPTS eat up all of our CPU and bandwidth. No PHP code is going to help prevent that. Just ultra smart dynamic filtering firewalls could do that. A thing hardly any of us bloggers has access to nor control over.
This is the worrying part. Not the crap appearing on our blog.
At 24 June '06 - 16:47 Lefebvre wrote:
I do not believe they really think that their spams will help the selling.
Maybe a .htacess rule can prevent it. Or not.
At 24 June '06 - 17:29 cargawar wrote:
I’m still n00b enough to be ignorant as I am about killing spammers: I wouldn’t know how to get rid of them.
This code you put in your PHP-files, does it work with Wordpress as well? And where do I put it? At the top or bottom of the page?
How about the PHP-files for plugins?
I’d appreciate an email about this if possible (in dutch will be fine)...
Cheers! Good topic!
At 24 June '06 - 21:12 Chris Mikkelson wrote:
The core (technical) problem is the botnets. Until we have every computer cleaned, behind a firewall, running a secure OS and applications, there will always be this platform just waiting to be taken advantage of by the kiddies. Since the bots usually run on end-user PCs, conveniently co-located with your readers. In other words, there’s no “port 80 filtering” analogue to port 25 filtering…
Keeping e-mail spam at bay, like USENET spam before then, requires the volunteer labor of a number of very sharp people. The situation will not be different for blogs. The main difference is that while both USENET and e-mail had been around for 10 years before the spammers took hold, blogs are a much younger and smaller (relative to e-mail, at least) scene which could be strangled in its infancy.
Worst case, public, unmoderated comments become useless before the anti-web-spam community organizes and starts taking a bite out of the problem. This seems unlikely but plausible, and frankly not too terrible for most blogs.
At 25 June '06 - 02:05 Marco wrote:
‘More secure operating systems’ aren’t gonna help prevent botnets. A lot of people say UNIX is more secure than Windows. This is only true to a certain extent. An OS is as secure as it’s admin. There’s so many hacked Linux boxes out there it’s not even funny anymore. And just imagine the whole world would be running Linux. Adding it to a botnet is as easy as sending Joe User an email stating: “run this as root. Really cool! You’ll see titties flapping all over your screen!”. Stupid people will happily do it, just like stupid windows users open every damn attachment they receive.
No, there’s a big responsibility here on the provider’s side of things. In this case, the providers that handle DSL/Cable internet connections for people. For example, some providers pro-actively prevent people from running SMTP servers, either on purpose or caused by a trojan that installed a bot. That means there won’t be any open relays on their network. And there’s much more to detect on the provider side. The funny thing is, the providers that don’t give a damn seem to come from the same earlier mentioned countries. And I fear one day some of those who administer the big networks are going to say: “Now it’s enough!” and start selectively allowing / blocking traffic.
@cargawar
On WordPress you should use Akismet. It will block 99.9% of all spam. When it comes to CPU usage: it really depends. In my case it was a horribly stupid spammer who thought my contact form was on all of those pages in order to submit a comment. Given the lack of any visible form ACTION in my code (it’s all AJAX and therefore hidden in the javascript) he thought comments can be posted here by POSTing to the actual page instead of the comment submit script.
What worried me the most about all of this is the fact that even though he was doing it all wrong, this spam run was targeted SPECIFICALLY at MY site. I have non-standard form fieldnames which he was actually trying to submit to. It means they really are getting smarter.
At 25 June '06 - 02:57 Max wrote:
I’ve actually blocked complete nations — Brazil and Korea were among the first — to keep the server up. E-mail, too. Of course, that’s just a temporary solution, and it doesn’t solve the 0wn3d-b0xx0rz problem. Only education could do that, and I’m very, VERY afraid that that’s just not going to happen…
At 26 June '06 - 01:29 Xslf wrote:
I don’t think people would spam if it didn’t work and bring in profit. However, considering how cheap it is to spam, I’m afraid that cutting off the profit line isn’t really an achiavble task
At 28 June '06 - 08:41 Montoya wrote:
At 28 June '06 - 09:21 cargawar wrote:
http://cargawar.com/blog/archives/1169
At 28 August '06 - 04:33 Edward Clarke wrote:
I’ve just gone to great lengths to successfully omit comment spam, using tools like MTKeywords and JavaScript authentication with email comment fallback but despite my blog now receiving an amazingly low amount of spam considering the vast amounts I previously encountered, attempts to spam are still hundreds per day according to my stats.
It doesn’t matter what method you use to access the web whether it’s an ADSL connection or a colocated server in a data centre, the onus is on the carrier to ensure fair use and no abuse.
Shame on them. I can understand that the majority of carriers invest solely on quality of service to the consumer but surely quality of the internet makes it a more attractive place to be which in turn drives demand.
A better business model perhaps?
At 06 November '06 - 01:01 Sebastiaan wrote:
Hopefully this comment wont attract more spammers to my site, as I only want to have a site for my own interests and a family communications channel!
At 23 November '06 - 11:38 macosbrain wrote:
At 18 December '06 - 06:03 Maurice Randall wrote:
I hope this isn’t seen as spamming this site here, but you might want to check out my new country blocking tools at fixingtheweb.com. I’ve had a method for using iptables in the Linux kernel for several months now, but that was only useful for a dedicated server and no good for a site running on a shared server. So, this past weekend, I completed a new package that works on a shared server. I hope it becomes handy for many, many of the better people in this world.
-Maurice
At 19 December '06 - 10:43 Xslf wrote:
After all, 98% of the comment spam in my blog is from the US, while I have very few “real” visitors from the US (as my blog is in Hebrew).
Somehow I have a feeling that isn’t what you had in mind when creating that tool…
At 07 March '07 - 02:56 Sarah Jones wrote:
You might be interested in htaccess code like
RewriteCond %{REQUEST_METHOD} POST
RewriteCond %{REQUEST_URI} !^/?(wp-comments-post|trackback|feed|login).php.*
RewriteRule .* – [F]
That only allows the POST method to be used on certain files, otherwise the request fails! This is better for you than using the php code because this won’t even load an instance of php!
http://www.askapache.com/2007/htaccess/2..
At 09 April '09 - 17:26 Alexander Ewering wrote:
An internet connection needs to cost $100 a month, PLUS $10 per GB of traffic used. This will eliminate effectively ALL SPAM, ALL viruses, and ALL worms as only people who actually NEED the internet will be able to use it.
All the morons downloading porn (and worms) all day long will be kept out.
Thank you :)