As part of my survey of SMTP servers on the Internet (a graphical representation of the results from that post are here), I need to find SMTP servers to survey. One of the ways that I've been doing that is I've been performing large numbers of DNS Mail eXchanger (MX) lookups and then probing the SMTP servers identified by those lookups. I haven't been able to perform those lookups on every domain registered, because not all registrars make their zone files available to researchers. I have a compendium of what I've learnt about zone file access agreements online if you're interested.
Specifically, I performed the following lookups:
| Zone | Number of lookups | |
| .arpa | 5 | |
| .asia | 9,044 | |
| .com | 72,529,657 | |
| .mobi | 819,849 | |
| .net | 10,734,157 | |
| .root | 281 | |
| 84,092,993 |
For each of these domains a DNS MX record lookup was performed using around 100 machines, and the results stored in a series of sharded tables in a MySQL database.
In aggregate, the results look like this:
| Total (IP, domain) tuples: | 72,863,506 |
| Total unique IPs: | 2,136,511 |
| Total unique domains: | 46,993,011 |
There are some interesting things to be found in the MX record data. For example, only 55.8% of the domains I scanned have an MX record at all. That might seem a bit counter intuitive, but when you take into account that a lot of domain names are unused or used simply for a web site, I guess its not that surprising. I would like to spend some more time verifying that this isn't a bug in my survey code, but I haven't gotten around to doing that yet.
Another interesting fact is that GoDaddy appears to be hosting a very large number of domains. Specifically, I found 12,105,590 domains which had one of just two IP addresses owned by GoDaddy as their MX record. That's 25.76% of all of my results. This means that's GoDaddy's domain hosting business is massive -- certainly much larger than I realized previously.
The IP addresses in question are 64.202.166.11 and 64.202.166.12. Some detail:
| IP | DNS Reverse |
| 64.202.166.11 | mailstore1.secureserver.net |
| 64.202.166.12 | smtp.secureserver.net |
secureserver.net is a domain registered to "Wild West Domains, Inc.", who appear to be part of the GoDaddy family (according to this GoDaddy help page, secureserver.net is used for GoDaddy DNS servers among other things). To determine how many of these domains are parked, I fired off some download jobs to download the top level page of each domain. At the moment, 1,087,885 of those downloads are complete.
Domains parked with GoDaddy HTTP 302 redirect from the top level page to a URL which is the domain name followed by a short identifier. For example, rastegarenterprises.net 302 redirects to rastegarenterprises.net/?bdb1d640 -- which is a page displaying advertising. Of the sites I have tested so far, 714,455 are parked in this manner.
That means GoDaddy currently has approximately 7,950,196 domains parked. That's around 9.4% of all the domains I have scanned!
Based on looking at IPs serving as MX for an unusual number of domains, the only other immediately obvious entry is that 184,213 domains point to 127.0.0.1. That seems a little bit odd to me.
I'm sure there is other interesting information in this MX data, but I think I'll leave it here for now.
Tags for this post: research(
) smtp()
posted at: 13:14 | path: /research/smtp | permanent link to this entry
John Ferlito
The MX to 127.0.0.1 makes some sense. In some cases its better than not having an MX. For example say I need to register a few hundred domains for trademark purposes. I point them all at my web server which is also my mail server. But I only want to receive mail from one of them.
Now for the ones that I don't want mail for if I don't have any MX at all and some tries to deliver mail to that domain then the MTA will do the A record lookup instead and try to deliver the mail there.
So now I have to deal with a whole heap more SMTP connections on my mail server from spammers. Setting it to 127.0.0.1 makes this problem go away.
Not an elegant solution and not one I'd recommend but its a solution