Solving a DNS Mystery on OS X

Someone on a forum I frequent noticed that telnet on Mac OS X 10.5 (Leopard) sometimes acts strangely when connecting to TCP port 25. Specifically, if the destination host is a DNS name then and that name has an MX record, telnet connects to the MX host instead of the named host.

For example:


$ host example.com
example.com has address 10.0.0.1
example.com mail is handled by 10 mail.example.com.
$ host mail.example.com
mail.example.com has address 192.168.1.1
$ telnet example.com 25
Trying 192.168.1.1...
Connected to example.com.
Escape character is '^]'.

Needless to say, this is a little weird. When the average person fires up telnet they want it to connect directly to the named host and port, not randomly start looking up MX records and connecting to that host instead. This isn’t the end of the world for interactive use (since you can look up the desired IP address manually and pass that to telnet), but it could be a problem for mail clients if it directed outgoing messages to the wrong host.

Since most of Darwin (the core of OS X, including most of the networking libraries and utilities), is open source, I decided to start investigating and see what I could find. The Darwin source for 10.5.2 is available here: Apple Mac OS X 10.5.2 Darwin sources (Index page with all releases here: Darwin sources)

First, I downloaded the telnet source to see if this was specific to telnet.

The relevant code in telnet (connecting to a specified port on a named host) is essentially:


commands.c:

struct addrinfo hints, *res, ...;

...

memset(&hints, 0, sizeof(hints));
hints.ai_family = family;
hints.ai_socktype = SOCK_STREAM;
hints.ai_flags = AI_CANONNAME;
error = getaddrinfo(hostname, portp, &hints, &res);

...

do {
printf("Trying %s...\n", sockaddr_ntop(res->ai_addr));
net = socket(res->ai_family, res->ai_socktype, res->ai_protocol);

if (connect(net, res->ai_addr, res->ai_addrlen) < 0)
{
struct addrinfo *next;

next = res->ai_next;

res = next;
}

connected++;

} while (connected == 0);

In short, it simply passes the specified host and port to getaddrinfo(3) and tries each resulting host address in order. So the MX address is coming from getaddrinfo(3), not telnet.

getaddrinfo is in Libinfo.

There, getaddrinfo hands off the query to its helper ds_getaddrinfo, which hands off the query to another helper LI_DSLookupQuery.


getaddrinfo.c:

static int
ds_getaddrinfo(const char *nodename, const char *servname, const struct addrinfo *hints, struct addrinfo **res)
{
...
status = LI_DSLookupGetProcedureNumber("getaddrinfo", &gai_proc);
...
request = gai_make_query(nodename, servname, hints);
...
status = LI_DSLookupQuery(gai_proc, request, &reply);
...
}


lu_utils.c:

LI_DSLookupQuery(int32_t procno, kvbuf_t *request, kvarray_t **reply)
{
...
status = libinfoDSmig_Query(_ds_port, procno, request->databuf, request->datalen, ilbuf, &illen, &oobuf, &oolen, &token);
...
}

This was something of a dead end. It looked like libinfoDSmig_Query was probably doing some RPC or IPC (looking up a function number by name at runtime and invoking the function indirectly) and the “DS” suggested it was related to DirectoryServices. However libinfoDSmig_Query wasn’t found as a name defined in any of the code I had so far, or the system headers. But it was a function, so it had to be defined somewhere. I jumped back to the top of lu_utils.c and found a reference to DSlibinfoMIG.defs.

Here we go. Definitely looks like an RPC/IPC API definition! The prefixes and function names must get concatenated by a preprocessor, explaining why grep wasn’t finding libinfoDSmig_Query anywhere.


DSlibinfoMIG.defs:

userprefix libinfoDSmig_;
serverprefix libinfoDSmig_do_;

...

routine GetProcedureNumber
(
server : mach_port_t;
name : proc_name_t;
out procno : int32_t;
ServerAuditToken bsmtoken : audit_token_t;
UserSecToken usertoken : security_token_t
);

routine Query
(
server : mach_port_t;
proc : int32_t;
request : inline_data_t;
out reply : inline_data_t;
out ooreply : pointer_t, Dealloc;
ServerAuditToken bsmtoken : audit_token_t;
UserSecToken usertoken : security_token_t
);

This still didn’t tell me where libinfoDSmig_Query was implemented. Assuming the “DS” meant DirectoryServices, I got the DirectoryServices source.

A search for libinfoDSmig_do_Query (concatenating the “routine” name onto the “serverprefix”) found:


ServerControl.cpp:

extern CCachePlugin *gCacheNode;

kern_return_t libinfoDSmig_do_Query(...)
{
...
returnedBuf = gCacheNode->ProcessLookupRequest(procnumber, request, requestCnt, aPID);
...
}

CCachePlugin::ProcessLookupRequest() was in CCachePlugin.cpp:


CCachePlugin.cpp:

kvbuf_t* CCachePlugin::ProcessLookupRequest ( int inProcNumber, char* inData, int inCount, pid_t inPID )
{
...
switch ( inProcNumber )
{
...
case kDSLUgetaddrinfo:
outData = DSgetaddrinfo( buffer, inPID );
break;
...
}
}

Getting warmer?


kvbuf_t* CCachePlugin::DSgetaddrinfo( kvbuf_t *inBuffer, pid_t inPID )
{
...

// we special case smtp and TCP combo and insert a specialized query
if( bHaveServiceName == true && bResolveName && (IPPROTO_TCP == protocol || 0 == protocol)
&& (SOCK_STREAM == socktype || 0 == socktype) && (strcmp(pService, "smtp") == 0 || strcmp(pService, "25") == 0) )
{
protocolListStr[serviceCount] = NULL;
protocolList[serviceCount] = "6";
socktypeList[serviceCount] = "1";
serviceNameList[serviceCount] = strdup( "MX" );
serviceCount++;
}

...
}

Boom! There you have it. Somebody hard-coded an exception into DirectoryServices to also look up the MX records of a host (in addition to IPv4 A and IPv6 AAAA records) when the destination port is TCP/25. As far as I can tell, there’s no way to turn that feature off (short of perhaps modifying the source and recompiling DirectoryServices).

Note: A quick Google search showed that “MIG” does indeed relate to IPC and RPC - it stands for “Mach Interface Generator”. See
Kernel Programming Guide - Mach Messaging and Mach Interprocess Communication.

ASPL for quoted ASPL code:
Portions Copyright (c) 1999-2003 Apple Computer, Inc. All Rights Reserved.

This file contains Original Code and/or Modifications of Original Code as defined in and that are subject to the Apple Public Source License Version 2.0 (the ‘License’). You may not use this file except in compliance with the License. Please obtain a copy of the License at http://www.opensource.apple.com/apsl/ and read it before using this file.

The Original Code and all software distributed under the License are distributed on an ‘AS IS’ basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND APPLE HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, QUIET ENJOYMENT OR NON-INFRINGEMENT. Please see the License for the specific language governing rights and limitations under the License.

Leave a Reply

You must be logged in to post a comment.