Technological Solutions 1, Legislative 0

Graph of spam since passage of the CAN-SPAM actAccording to "The New York Times":http://www.nytimes.com/2005/02/01/technology/01spam.html?ex=1265000400&en=7f39d17819d9ca0f&ei=5090&partner=rssuserland (use "BugMeNot":http://bugmenot.com/ if it prompts you to log in), the amount of spam has increased to an all time high in the year since the passage of the CAN-SPAM act. At best, the act has done little to nothing to curb spam on the internet. However, it is more likely that CAN-SPAM has actually increased the amount of spam being delivered because it effectively legalized spam.

I'm not really here to talk about how ineffective legislative solutions are. Rather, I'd like to talk about how technology has reduced the amount of spam I receive at my multiple email addresses (one of which I've had since 1996) to near zero, without my ever obscuring my email address when I post it on the Internet.

Originally, I only used DNS-based realtime blackhole lists, which list the IP addresses of computers known to send spam. Unfortunately, this technique is very sledgehammer-like, and it is pretty much guaranteed to produce false positives if you use some of the "looser" blacklists, and it won't catch much more than 50-60% of spam if you only use conservative blacklists that don't generate false positives. My policy for choosing DNSBLs is one of zero tolerance: one false positive means I don't use it any more. The nice thing about only using DNSBLs is that I never had to deal with a "Junk" folder.

While I was using only the DNSBLs, several spams a day would make it to my inbox. I'd report these to the various ISPs via "SpamCop":http://www.spamcop.net/, and I used SpamCop's DNSBL. I eventually realized that reporting spam to ISPs was a lost cause, if not harmful, and that the SpamCop proprietors were involved in lobbying for spam legislation, so I stopped using SpamCop. For a while, I used only "Bogofilter":http://bogofilter.sourceforge.net/, but that requires a lot of training, and it's limited in that it can only use local knowledge of past spams and legitimate emails to make its decisions.

I also tried out "TMDA":http://tmda.net/ for a while, which uses challenge/response mail authentication. Some have touted challenge/response as the only/ultimate solution to spam. As I realized very quickly, not only is it not a solution to spam, but it mainly serves to "piss people off":http://jeremy.zawodny.com/blog/archives/001931.html. Plus, it's likely to drop mails coming from automated sources. At this point, if I get a challenge from "EarthLink spamblocker":http://www.earthlink.net/spamblocker/ or "MailBlocks":http://www.mailblocks.com/ (which has been acquired by AOL, meaning I'll be getting even more of these things! Ugh!), I just delete it. Or, if I really want to get in touch with the user, I'll respond to the challenge and then send them a nasty note.

Eventually, I had to start using "SpamAssassin":http://spamassassin.apache.org/ along with only the most conservative DNSBLs: I reject mail from servers listed in the conservative blacklists and pass the rest through SpamAssassin, sending anything with a score of 5.0 or greater into my Junk folder. This has worked well for quite a while, with about 50 spams a day being delivered to my Junk folder, and two or three getting into my inbox, with about one legitimate mail a month being mistakenly delivered to the Junk folder.

Checking my Junk folder every couple of days became tedious (I'm very lazy), so I started throwing everything with a SpamAssassin score of 18 or higher straight into the trash. Later, I started doing this with suspected spams sent to addresses I didn't care about losing some mails for, like the various webmaster accounts I handle. This reduced the amount of spam in my junk folder quite a bit.

Even though at this point the amount of spam in my inbox (still a couple per day) and the amount I had to look through in my junk folder (25 or so per day) was quite manageable, I decided to try a technique that had been around for a couple years but that I'd never tried: "greylisting":http://www.greylisting.org/. The principle behind greylisting is simple: keep track of sending host/sender email address/recipient email address triplets, and reject a message with a temporary failure code the first time you see a triple. It turns out that most spam sending programs and worms won't bother retrying failed messages, so reduced the amount of spam even making it into my Junk folder by about 90% right away!

Of course, spammers adapt, and if greylisting becomes even more widespread, more spammers will just retry failed messages, but greylisting has a side effect: it delays mail from sources it hasn't seen before. This allows the blacklists, checksum databases, etc. to "discover" new spam runs before the message even makes it to spamassassin! In addition, I have several spamtrap addresses in my various domains, and I always deliver email to those spamtraps immediately without greylisting it, so that it can be auto-reported as quickly as possible.

This combination of techniques has reduced the amount of spam that makes it to my Junk folder to about 2-3 per day, the amount of spam that used to make it to my inbox, and the amount of spam in my inbox is now down to near zero. In fact, none has made it through since I started greylisting, probably because the delay helps SpamAssassin make better decisions. It does this without inconveniencing the sender at all. Well, it does inconvenience their mail server slightly the first time they try to send me mail. But the user will likely never know this has happened.

If these techniques work so well for someone with old, widely-posted, unobfuscated email addresses, imagine how they work for regular joes! Of course, most people need an ISP with some smarts to implement this stuff for them, but at this point, if you get too much spam, it's not because of the lack of a law or because the filtering technology isn't there. It's because you (or your ISP) aren't using the appropriate technology to filter your mail.

Share this

Ever since I installed the

Ever since I installed the Vipul's Razor plugin to Spamassassin, I have had minimal spam in my inbox and 0 false positives.

I use Mozilla Thunderbird.

I use Mozilla Thunderbird. It works fine for me.