Vlad Mazek – Vladville Blog

Yesterday we implemented some new magic against tar-pitting smashers and man does it make a difference: Since noon EST we have rejected an average of 46,128 messages per-node. We then ran a report – 100% of the IP addresses that had their email rejected had mail delivered to the SureSPAM Quarantine the day before.

The significance? System performance, employee efficiency.

System Performance – Any bit helps. By rejecting the messages at the perimeter (yes, even ExchangeDefender as a perimeter service has a perimeter network) we save network resources because we have less messages to process through bayesian filters, less statistics to calculate and ultimately less spam that we’ll have to store.

Employee Efficiency – Two fold: First and less important, the remote likelyhood that the mail produced by these spamming hosts does not get properly identified as SPAM and ends up in the users inbox. Second and most important: less mail for the employees to review. Since I can guarantee that no legitimate mail was sent by these systems I can save my customers time from having to browse through meaningless quarantine reports and let them look at the stuff that is actually likely to be SPAM.

That last part is where I and some of my partners have very vocal arguments:

My mission is to cut down and eliminate spammers and their impact to the customer, not create pretty graphs.

My partners need the graphs and the data to prove to their customers that the SPAM is really bad enough to justify ExchangeDefender.

I think pretty much anyone who would spend a day on IMF/Outlook-only protection would scream their lungs off and rip their hair out over the amount and volume of junk thats floating out there. And while I can understand that sometimes people need to be shown what they are paying for I am not a fan of selling the fear. People either have a problem with their email or they don’t. And I’ll tell you this much – we do not lose customers. Nobody that has ever become a paying ExchangeDefender customer has ever left (short of AUP violations and going out of business) so I’m doing the right thing here, I really don’t think that at the end of the day the customer is going to be concerned about the value if they see 0 junk in their mailbox – they all have multiple accounts and contacts that are being flooded, they know the impact.

The second and more techical reason for what I do — By rejecting a message on connection basis I am saving back-scatter. When I reject the message with an SMTP error code the remote mail server is left holding the bag. When I accept the message and then have to send it back due to the prohibited content or system policy I am doubling the amount of junk that is being sent over the Internet. “But Vlad, why don’t you just not send it at all, just junk it?” – supose the message was legit but did not meet the policy requirements, or had questionable content, or did not meet the security requirement (some customers only accept encrypted messages or only specific senders), or had a virus, or had an attachment that customer would not accept, or, or, or.. My whole technical point here is that there are many good reasons for sending back an NDR and we (ISPs) need to be good Internet citizens and not flood the network with the meaningless ones.

Earlier today we have made some significant changes to the tar-pitting mechanism under ExchangeDefender. The new mechanism is designed to reject messages from hosts that do not follow the proper RFC SMTP dialog and attempt to smash tar-pitting. More on the basic concept of tar-pitting is described here.

The Problem

While tar-pitting is great for throttling remote mail servers and reducing their ability to efficiently deliver a lot of messages, the concept only applies against botnet servers that are attempting to deliver mail in bulk. Anotherwords, tar-pitting is only effective against servers that are concerned about getting the message out as fast as possible. By delaying the SMTP greeting banner, in theory, the remote mail server would have to wait a pre-determined amount of time before starting to send mail. Many open connections at once would overload a single node.

However, spammers no longer exclusively use single nodes in a full force attack. They use the botnet concept by load balancing their broadcasts through multiple servers. As such, those servers connect every few minutes and only relay a single message. By doing so its hard to blacklist them immediately because their overall reputation does not have enough data to be determined. These botnets are designed to bypass tar-pitting by opening a connection and sending data as soon as the connection is opened.

The conversation looks somewhat like this:

Trying xx.xx.xx.xx.exchangedefender.com.

Escape character is ‘^]’.
ehlo spamming-idiot.org
mail from: spammer@spammer.org
rcpt to: vlad@ownwebnow.com
data
Subject: Get a college diploma.
Ohio State University may be a loser but they’ll give you a Ph.D in nuclear physics based on your life experience.
.

Now the (target) tar-pitting mail server has accepted the connection but it has never sent the SMTP greeting. However, it will process the message as soon as its tar-pitting interval passes, thus in part bypassing the tar-pitting and delivering the message. Not good.

Notice that the client above did not wait for the 200 greeting banner, did not wait for the 250 Hello, did not wait for the server to acknowledge the recipient and the sender. They just wrote to the socket and waited. Now even though this does diminish the spammers performance a little (by taking 5 seconds to deliver the message) the message still gets delivered. That’s a problem.

The Solution

The solution is fairly simple: Drop connections with mail servers that are not adhering to RFC. The second the mail server issues a command before the 2.2.0 hostname greeting banner it will get dropped, logged and its tar-pitting interval extended.

Instead of a tar-pitting process that delays the connection a few seconds, this process allows for a connection immediately but delays the SMTP greeting banner a few seconds. As such, it can eliminate server load caused by spammers that think they have found a way around tar-pitting.

We ran this in testing on our production systems and have found 0 false positives over the course of one week. All hosts that were rejected were also on multiple RBLs. The implementation is transparent to the user and administrator and introduces a random (less than 5) second pause on all connections that do not have a reputation rating with ExchangeDefender. Less spam, less stuff to review, less bandwidth and less stress for you.