What you need to know about IP address blacklisting

Having your IP address end up on a so called ‘blacklist’ can be a troublesome experience, especially when not anticipated. In most cases, it is a sign that something is wrong on the server(s) you rent or own, or that maybe one of the end users hasn’t followed email sending guidelines. This post is dedicated to those who want to know more about IP address reputation and what can be done to resolve issues identified by other parties.

The ongoing fight against spam

As we have already explained in the Spam blogs (I and II), email spam continues to be an issue. Due to the ever-evolving problem of email spam, there is an understandable need to have measures to combat this. Over the years, several efforts have been made to prevent unsolicited emails from reaching email inboxes by a plethora of means. Many of these proposed solutions have had promising technical white papers but few have actually resulted in an implementation that is either scalable, reliable or both.

What is a blacklist (or DNSBL)?

Nowadays, practically speaking, the most useful identifiers to help with stopping spam en masse are the IP addresses of the servers that emit the unsolicited messages. Thus, the prevention mechanism most often employed by mailserver administrators is a simple block of these ‘bad’ IP addresses. In order to create efficiency in this process the idea of crowdsourcing this data and centralizing it was fostered and ‘DNS-based Blackhole Lists’ (DNSBLs) were born.

DNSBLs are in some way a form of internet police, the “internet sheriffs” you might say. If an IP address gets involved with something the DNSBL operators disapprove of, and they become aware of this, they might decide to put that IP address on their list.

How do IP addresses end up on a DNSBL?

The thing with DNSBLs is that each one of them operates within its own set of rules and with a focus on a certain abuse category. The most common abuse category among these lists is obviously spam but there are also blacklists that focus on hacking, malware, botnets or even Tor exit nodes. The various DNSBLs employ a wide variety of techniques to gather these IP addresses including: mailtraps, honeypots, botnet analysis and crowdsourcing data from participating mail clients.

How do IP address lookups work?

As mentioned already, each DNSBL has its own criteria for designating an IP address as having a bad reputation. This reputation is published by means of a DNS record and the DNS servers run by DNSBL administrators are open to the public to perform lookups of IP addresses on. DNS was originally meant for looking up domain names but it has become the de-facto method to distribute IP address reputation designations due to its low overhead and high scalability.

From a technical perspective, a lookup is done by performing the following steps:

1. Reverse the IP address
2. Append the DNSBL domain
3. Do a DNS lookup of the resulting ‘domain’

This will either result in ‘NXDOMAIN/Non-existent domain’ response or will return an IP address (usually in the 127.0.0.x range). When an IP address is returned, the IP address is ‘listed’. Below, you will find an example of each:

‘Blacklisted’ IP address (1.54.110.152):

Name:      152.110.54.1.zen.spamhaus.org
Address:  127.0.0.10

‘Clean’ IP address (8.8.4.4):

Name:      4.4.8.8.zen.spamhaus.org
*** server can’t find 4.4.8.8.zen.spamhaus.org: Non-existent domain

Most DNSBLs have guidelines on how to use the responses from their DNS server. In general, it is advised to use data from multiple sources before blocking emails. However, many mail servers are knowingly or unknowingly set up to refuse emails from any IP address that is on at least one DNSBL. While not ideal and often not according to guidelines, this is the reality that email senders have to live with; a single, potentially false positive listing can have disastrous results on email deliverability.

Listed, now what?

Once an IP address is listed on a DNSBL, for whatever reason, there is a chance that email deliverability will be affected. This is a problem that needs to be resolved. Luckily, most lists allow for de-listing once the operator of the IP address has confirmed a solution to the problem or incident that caused the listing. An example of a de-listing request form can be found on Barracuda Central’s website. Just as the criteria for listing an IP address differ from DNSBL to DNSBL, the requirements for de-listing are also list-specific. However, in most cases, de-listing requests are processed within 24 hours.

There is one thing that most DNSBLs have in common: the way they deal with removal requests while the source of the problem is NOT taken care of. Often, this will result in more difficulty getting the listing removed in future requests. While mitigation of the cause would initially have been enough to get an IP address de-listed, after invalid removal requests, the DNSBL might now require you to provide additional proof of the resolution. It is thus wise to only request de-listings when you are sure that the problem has actually been resolved.

What about Hotmail/Microsoft?

If you have mail delivery issues to Microsoft managed domains, it might be because Microsoft is bouncing your emails, if this is the case, you will get the following response from the destination mail server:

“host mx4.hotmail.com[xx.xx.xx.xx] said: 550 SC-001 Mail rejected by Windows Live Hotmail for policy reasons. Reasons for rejection may be related to content with spam-like characteristics or IP/domain reputation problems. If you are not an email/network admin please contact your E-mail/Internet Service Provider for help. Email/network admins, please visit MSN Postmaster for email delivery information and support (in reply to MAIL FROM command)”

Microsoft takes a different approach to preventing spam. The above message doesn’t necessarily mean that your specific IP address is ‘blacklisted’. Lately, more and more ranges are ‘listed’ by default. While the above bounce might indicate otherwise, Microsoft has effectively taken a ‘whitelist’ approach for email delivery originating from certain ranges to their platform. Simply said, Microsoft wants to know what type of email you send before you can send email to their managed inboxes. A request to be whitelisted can be made on this page.

While the Microsoft list is not publicly available, you can request to have access to your IP address status through Microsoft’s Smart Network Data Service.

And Leaseweb?

As Leaseweb offers unfiltered access to the internet, like any other large unmanaged hosting provider, it cannot always prevent the negative effects on network reputation by intentional and unintentional unsolicited – or even malicious – network activity. To mitigate these issues, in addition to actively monitoring our network reputation, we also put effort in educating our customers because, after all, we can only create a safer internet with the collaboration of our customers.

When we identify new issues within our network we do our best to mitigate these as quickly as possible. To facilitate this, we use every available information source. To support the DNSBL community, we have included several in our Community Outreach Program, a notable one is Spamhaus.

If you run a medium to large sized DNSBL, we are happy to help you out as well by providing free servers for additional mirrors!