UDP广播接收问题与Ubuntu 9.04，但不是8.04

更新

01-27ter：添加了rp_filter信息
01-27之二：请注意，9.04框在不同的界面上工作。
01-27：增加接口configuration信息和分析报文。

原始post

我有两个非常相似的硬件configuration（SuperMicro 1U系统，包含双Xeon CPU和两个以太网端口），一个运行Ubuntu 8.04（Linux 2.6.24-26-服务器），一个运行Ubuntu 9.04（Linux 2.6.28- 17-服务器）。这两个eth1连接到其他各种服务器向各个端口发送广播UDP数据包的同一networking。在两台主机上，使用eth1上的tcpdump，我可以看到这些广播的UDP数据包到达。

然而，在8.04盒子上，我可以有一个简单的程序听他们就好了，在9.04盒子上一个相同的程序永远不会收到它们。作为一个高层次的概述，下面是一个示例Haskell程序，它可以在一个上运行，而不是在另一个上运行（在两者上使用相同版本的GHC）：

import Network.Socket port = 5515 main :: IO () main = do do sock <- socket AF_INET Datagram defaultProtocol bindSocket sock $ SockAddrInet (fromIntegral port) iNADDR_ANY loop sock where loop sock = do msg <- recv sock 2048 print msg loop sock

如果这个问题在GHC中碰巧是非常奇怪的（虽然它是两个相同的版本），我写了一个C程序来做同样的事情：

 #include <arpa/inet.h> #include <netinet/in.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #include <sys/types.h> #include <sys/socket.h> #include <unistd.h> #define BUFLEN 512 #define NPACK 10 #define PORT 5515 void diep(char *s) { perror(s); exit(1); } void showb(int s) { int val, len, retval; len = sizeof(val); retval = getsockopt(s, SOL_SOCKET, SO_BROADCAST, &val, &len); printf("showb retval=%d val=%d\n", retval, val); } int main(int argc, char **argv) { struct sockaddr_in si_me, si_other; int s, i, slen=sizeof(si_other); char buf[BUFLEN]; if ((s=socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP))==-1) diep("socket"); showb(s); i = 1; if (setsockopt(s, SOL_SOCKET, SO_BROADCAST, &i, sizeof(i))==-1) diep("setsockopt"); showb(s); memset((char *) &si_me, 0, sizeof(si_me)); si_me.sin_family = AF_INET; si_me.sin_port = htons(PORT); si_me.sin_addr.s_addr = htonl(INADDR_ANY); if (bind(s, &si_me, sizeof(si_me))==-1) diep("bind"); puts("Listening."); for (i=0; i<NPACK; i++) { if (recvfrom(s, buf, BUFLEN, 0, &si_other, &slen)==-1) diep("recvfrom()"); printf("Received packet from %s:%d\nData: %s\n\n", inet_ntoa(si_other.sin_addr), ntohs(si_other.sin_port), buf); } close(s); return 0; }

你会注意到，在这种情况下，为了好玩，我也打开了套接字上的SO_BROADCAST标志，并确认它被打开了，尽pipe这对程序的行为没有什么不同。即使将build立在8.04上的二进制文件复制到9.04箱，反之亦然，在所有情况下，运行在8.04箱上的程序都会看到UDP广播数据包，而9.04箱却不会。

我究竟做错了什么？

更新01-27：

这里是工作（8.04）主机的ip link和ip ether的输出：

 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1362 qdisc pfifo_fast qlen 1000 link/ether 00:30:48:d3:4b:06 brd ff:ff:ff:ff:ff:ff 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 100 link/ether 00:30:48:d3:4b:07 brd ff:ff:ff:ff:ff:ff 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1362 qdisc pfifo_fast qlen 1000 link/ether 00:30:48:d3:4b:06 brd ff:ff:ff:ff:ff:ff inet 192.168.228.130/28 brd 192.168.228.143 scope global eth0 inet6 fe80::230:48ff:fed3:4b06/64 scope link valid_lft forever preferred_lft forever 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 100 link/ether 00:30:48:d3:4b:07 brd ff:ff:ff:ff:ff:ff inet 172.40.4.130/24 brd 172.40.4.255 scope global eth1 inet6 fe80::230:48ff:fed3:4b07/64 scope link valid_lft forever preferred_lft forever

而对于非工作（9.04）服务器：

 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1362 qdisc pfifo_fast state UP qlen 1000 link/ether 00:30:48:d9:38:da brd ff:ff:ff:ff:ff:ff 3: eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000 link/ether 00:1b:21:36:19:fd brd ff:ff:ff:ff:ff:ff 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 100 link/ether 00:30:48:d9:38:db brd ff:ff:ff:ff:ff:ff 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1362 qdisc pfifo_fast state UP qlen 1000 link/ether 00:30:48:d9:38:da brd ff:ff:ff:ff:ff:ff inet 192.168.228.132/28 brd 192.168.228.143 scope global eth0 inet6 fe80::230:48ff:fed9:38da/64 scope link valid_lft forever preferred_lft forever 3: eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000 link/ether 00:1b:21:36:19:fd brd ff:ff:ff:ff:ff:ff 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 100 link/ether 00:30:48:d9:38:db brd ff:ff:ff:ff:ff:ff inet 172.40.4.132/24 brd 172.40.4.255 scope global eth1 inet6 fe80::230:48ff:fed9:38db/64 scope link valid_lft forever preferred_lft forever

请注意，对于这两种情况，eth1是广播到达的端口。

下面是一个程序没有收到的示例广播包的完整解码（来自非工作的9.04服务器上的tshark）：

 Frame 193555 (271 bytes on wire, 271 bytes captured) Arrival Time: Jan 25, 2010 08:00:00.535345000 [Time delta from previous captured frame: 0.001508000 seconds] [Time delta from previous displayed frame: 0.000000000 seconds] [Time since reference or first frame: 6590.956186000 seconds] Frame Number: 193555 Frame Length: 271 bytes Capture Length: 271 bytes [Frame is marked: False] [Protocols in frame: eth:ip:udp:data] Ethernet II, Src: Cisco_aa:c0:28 (00:d0:bb:aa:c0:28), Dst: Broadcast (ff:ff:ff:ff:ff:ff) Destination: Broadcast (ff:ff:ff:ff:ff:ff) Address: Broadcast (ff:ff:ff:ff:ff:ff) .... ...1 .... .... .... .... = IG bit: Group address (multicast/broadcast) .... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default) Source: Cisco_aa:c0:28 (00:d0:bb:aa:c0:28) Address: Cisco_aa:c0:28 (00:d0:bb:aa:c0:28) .... ...0 .... .... .... .... = IG bit: Individual address (unicast) .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default) Type: IP (0x0800) Internet Protocol, Src: 192.166.1.120 (192.166.1.120), Dst: 255.255.255.255 (255.255.255.255) Version: 4 Header length: 20 bytes Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00) 0000 00.. = Differentiated Services Codepoint: Default (0x00) .... ..0. = ECN-Capable Transport (ECT): 0 .... ...0 = ECN-CE: 0 Total Length: 257 Identification: 0xfad3 (64211) Flags: 0x04 (Don't Fragment) 0... = Reserved bit: Not set .1.. = Don't fragment: Set ..0. = More fragments: Not set Fragment offset: 0 Time to live: 252 Protocol: UDP (0x11) Header checksum: 0xc0f9 [correct] [Good: True] [Bad : False] Source: 192.166.1.120 (192.166.1.120) Destination: 255.255.255.255 (255.255.255.255) User Datagram Protocol, Src Port: 56172 (56172), Dst Port: 5515 (5515) Source port: 56172 (56172) Destination port: 5515 (5515) Length: 237 Checksum: 0x01ba [correct] [Good Checksum: True] [Bad Checksum: False] Data (229 bytes) 0000 41 37 30 33 34 30 38 30 30 30 30 30 30 31 31 30 A703408000000110 0010 4b 52 53 50 49 4f 50 4b 32 49 4b 52 34 32 30 31 KRSPIOPK2IKR4201 0020 45 32 32 32 35 33 30 30 32 31 30 30 30 30 30 30 E222530021000000 0030 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 0000000000000000 0040 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 0000000000000000 0050 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 0000000000000000 0060 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 0000000000000000 0070 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 0000000000000000 0080 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 0000000000000000 0090 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 0000000000000000 00a0 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 0000000000000000 00b0 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 0000000000000000 00c0 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 0000000000000000 00d0 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 0000000000000000 00e0 30 30 30 30 ff 0000. Data: 413730333430383030303030303131304B525350494F504B...

我已经从工作8.04服务器上的转储对这个相同的数据包进行了区分，并且数据包本身是identicial; 唯一的区别是在帧数（pcap文件内）和数据包接收时间（1.224毫秒的差异，鉴于两个主机使用相同的NTP服务器，但并不完全不合理）。

更新01-27之二

我已经做了进一步的实验，在8.04主机上生成我自己的广播数据包，并将它们发送到9.04主机，而9.04主机在8.04主机发送数据时正好接收数据包，它们到达eth0或eth1。

更新01-27ter

 net.ipv4.conf.all.rp_filter = 1 net.ipv4.conf.default.rp_filter = 1 net.ipv4.conf.eth0.rp_filter = 0 net.ipv4.conf.eth1.rp_filter = 0 net.ipv4.conf.lo.rp_filter = 1

而在9.04上的主机是：

 net.ipv4.conf.all.rp_filter = 1 net.ipv4.conf.default.rp_filter = 1 net.ipv4.conf.eth0.rp_filter = 1 net.ipv4.conf.eth1.rp_filter = 1 net.ipv4.conf.eth2.rp_filter = 1 net.ipv4.conf.lo.rp_filter = 0

所以问题是net.ipv4.conf.eth1.rp_filter sysctl设置。当它在8.04盒子上设置为0时，我正在进行松散的反向路径检查，这意味着一个数据包可以从任何可以在任何接口上路由到的目的地进入。在9.04盒子上，我正在进行严格的检查，这意味着如果对这些数据包的回复会从不同的接口出来，它将拒绝到达接口的数据包。

到达eth1到255.255.255.255的数据包是我不应该接收的数据包，因为255.255.255.255是本地网络广播地址，但是这些数据包的来源不在本地网络。所以本质上，在我收到一个提要的网络上配置错误，我必须处理这种错误配置。