<< | Thread Index | >> ]    [ << | Date Index | >> ]

To: cipe-l,AT,inka,DOT,de
Subject: Detalied analysis of UDP recv refused and query about fixing it
From: Len Reed <lreed,AT,linuxcare,DOT,com>
Date: Tue, 09 Dec 2003 12:57:37 -0500
Organization: Linuxcare, Inc.

Hi all,

I'm seeing UDP connection refused. I see complaints about this in the mailing list archives, but I couldn't find a detailed analysis or fix. I'm posting my analysis here; please let me know if there's a fix or you'd like to help me craft one. I know network programming but am unfamiliar with cipe internals.

The problem is that, after pkcipe successfully negotiates the "connection", the server end refuses to accept UDP datagrams.

From the client's syslog:

ciped-cb[3789]: kxchg: recv: Connection refused

My setup is:
0. Both ends running Linux, cipe 1.5.4.
1. pkcipe client behind a gateway that is doing NAT. (In my case the client is single-homed if you don't count the cipe point-to-point.)
2. pkcipe server is multi-homed. The outward-facing NIC handles the Internet traffic; an inward-facing NIC routes traffic to/from the private 10.*.*.* networks.


Everything below I learned from netstat, route, tcpdump, etc., plus the cipe config files. I haven't dug into the source code yet.

If the client's eth0 address is 192.168.x.y, it *always* works. If the client's eth0 address is 10.0.0.z, it almost never works. (Open mystery: "why does it work every once in a while?")

The failure mode is that pkcipe negotiates fine, but the server refuses the UDP packets, returning ICMP connection refused. Running nestat shows that problem is that the cipe process that should be listening for those UDP packets is bound to the wrong interface! It's bound to the inward-facing interface! When the datagrams show up on the correct interface, they get rejected.

It appears that the server is using the client's local, unNATted IP address when binding the UDP port. If that address is in 10.0.0.0/8, it gets bound to the inward facing (wrong) LAN. If the client is in 192.168.0.0/16, though, it gets bound correctly. (I presume that this is, too, is luck rather than proper programming; with no explicit route to the client, it binds to the default route, which happens to be the correct interface.)

A supporting clue can be found in /var/run/cipe/<user> on the server. This contains the IP address of the client. Instead of the public Internet address, which is what I'd expect to find there, it contains the private unNATted address. I can see no reason for the server to know this address or care about it. (In fact, in the general case, a machine doesn't have a single address. If the client host had three NICs, which IP address would be chosen? The one that routes to the NAT interface? Beats me. IMO all answers are wrong. These addresses should be hidden from the server.) Furthermore, for it to even know the private IP address, that address had to be passed as data during the pkcipe negotiation; the IP headers of course contain the NATted (public) address.

So, solutions could include

1. Accepting UDP on any interface. Since there are session keys in force, I don't see that this would be a security hole. It doesn't seem quite as nice as #2.

2. Fix it to bind to the correct interface. That interface is the one that pkcipe connection came in on.

An alternate #2 might involve making /var/run/cipe/<user> contain the public, not private address of the client. Correct behavior might just "fall out" then. (As I said, using the private address just seems wrong.) There may be compatibility implications w.r.t. existing cipe clients if doing this.

Let me know if there's a solution or workaround for this; I need to fix this in the next couple of days so I'll be digging into the code soon enough.

Oh, one last note. Someone reading this is going to mistakenly think that the routes overlap and so this is in an unsolveable problem. Not so. Technically, the inward facing 10.0.0.0/8 does overlap the 10.0.0.0/24 that my hotel put me on. (Now you see why I want to solve this now; it works fine from home, where I use 192.168, but the hotel isn't giving me that choice.) However, I don't need to get to anything in the 10.0.0.0/24 range behind the server, so it doesn't matter. In fact, in the worst case, I ought to be able to set up my laptop's routing so that I can get to anything in 10.0.0.0/8 at the other end except (a) the address my laptop is using on the hotel subnet and (b) the default gateway on that subnet. (There's nothing else on the hotel LAN I care about; I'm too busy to try to hack into other hotel guests' laptops. :)) Dealing with (a) or (b) would be very ugly; maybe I could use some iptables magic to deal with that.

Len


<< | Thread Index | >> ]    [ << | Date Index | >> ]