Linux:发送以太网帧时,正在重写ethertype

Question

我编写了一个C程序,它将以太网帧直接写入线路(以两种模式运行,发送器或接收器).发送方正在发送带有两个VLAN标记的帧(QinQ),但奇怪的是,当帧到达接收方时,ethertype已更改为标准(单个)VLAN封装帧的类型.NIC是否有可能这样做,或者Linux不允许这样做？Wireshark显示与tcpdump相同的行为.

为了解释下面的图像,发送器正在向以太网广播地址FF:FF:FF:FF:FF:FF发送帧以找到接收器(这些是通过交叉电缆连接的两台测试机,但下面的结果与开关或集线器).正如您所看到的那样,帧上有两个VLAN标记,外部标记的ethertype为0x8100,VLAN ID为40,内部VLAN的ethertype为0x8100,VLAN ID为20.我们都知道,使用QinQ帧时,外框应该具有0x88a8的ethertype!

当在我的应用程序中从发送方发送帧时,它们的外部ethertype为0x88a8,但根据下图,它们在内部和外部ethertypes上都接收到0x8100.突出显示的文本是接收者发送回复,因为您可以看到帧在外框上有0x88a8,在内部有0x8100.另一台机器上的tcpdump显示相同的内容(它是相同的代码!帧内部发送0x88a8外部0x8100但总是接收为0x8100外部和0x8100内部).

void BuildHeaders(char* &txBuffer, unsigned char (&destMAC)[6], 
     unsigned char (&sourceMAC)[6], short &PCP, short &vlanID,
     short &qinqID, short &qinqPCP, int &headersLength)
{

int offset = 0;

short TPI = 0;
short TCI = 0;
short *p = &TPI;
short *c = &TCI;
short vlanIDtemp;

// Copy the destination and source MAC addresses
memcpy((void*)txBuffer, (void*)destMAC, ETH_ALEN);
memcpy((void*)(txBuffer+ETH_ALEN), (void*)sourceMAC, ETH_ALEN);
offset = (ETH_ALEN*2);

// Add on the QinQ Tag Protocol Identifier
vlanIDtemp = qinq
TPI = htons(0x88a8); //0x88a8 == IEEE802.1ad, 0x9100 == older IEEE802.1QinQ
memcpy((void*)(txBuffer+offset), p, 2);
offset+=2;

// Now build the QinQ Tag Control Identifier:
TCI = (qinqPCP & 0x07) << 5;
qinqID = qinqID >> 8;
TCI = TCI | (qinqID & 0x0f);
qinqID = vlanIDtemp;
qinqID = qinqID << 8;
TCI = TCI | (qinqID & 0xffff);

memcpy((void*)(txBuffer+offset), c, 2);
offset+=2;

// VLAN headers
vlanIDtemp = vlanID;
TPI = htons(0x8100);
memcpy((void*)(txBuffer+offset), p, 2);
offset+=2;

TCI = (PCP & 0x07) << 5;
vlanID = vlanID >> 8;
TCI = TCI | (vlanID & 0x0f);
vlanID = vlanIDtemp;
vlanID = vlanID << 8;
TCI = TCI | (vlanID & 0xffff);

memcpy((void*)(txBuffer+offset), c, 2);
offset+=2;

// Push on the Ethertype (IPv4) for the payload
TPI = htons(0x0800);
memcpy((void*)(txBuffer+offset), p, 2);
offset+=2;

headersLength = offset;

}

sendResult = sendto(sockFD, txBuffer, fSizeTotal, 0, (struct sockaddr*)&socket_address, sizeof(socket_address));

Answer 1

(Fully rewritten to simplify the answer. I also fixed quite a few bugs in my C header and source files listed below.)

There was a discussion about exactly this on the linux-netdev mailing list in April 2014, subject "802.1AD packets - Kernel changes ether type from 88A8 to 8100 on all packets".

It turns out that the kernel does not change the ether type, it simply consumes it on receiving a packet. I show below that it is correctly used for VLAN routing (including separate rules for 802.1AD and 802.1Q VLANs), given a recent enough kernel. Even if the VLAN tag is not used for routing (say, if there are no VLANs configured, or if the 8021q kernel module is not loaded), the VLAN tag is consumed by the kernel.

因此,原始问题"当发送以太网帧时正在重写ethertype"时,是不正确的:ethertype没有被重写.它由内核使用.

因为内核使用了VLAN标记,所以libpcap是tcpdump使用的数据包捕获库,wireshark等人. - 尝试将其重新引入数据包标头.不幸的是,它总是使用802.1Q VLAN头(8100).

有一个建议的更改 libpcap,修复了libpcap中的这个问题,但在撰写本文时,似乎还没有包含它; 你仍然可以htons(ETH_P_8021Q)在Linux的libpcap源文件中的几个地方看到硬编码.

我不能假设你会接受我的话,所以让我告诉你如何为自己确定这一点.

让我们编写一个简单的数据包发送器和接收器,它直接使用内核接口,而无需libpcap的帮助.

rawpacket.h:

#ifndef   RAWPACKET_H
#define   RAWPACKET_H
#include <unistd.h>
#include <sys/socket.h>
#include <sys/ioctl.h>
#include <netpacket/packet.h>
#include <net/ethernet.h>
#include <net/if.h>
#include <arpa/inet.h>
#include <linux/if_ether.h>
#include <string.h>
#include <errno.h>
#include <stdio.h>

static int rawpacket_socket(const int protocol,
                            const char *const interface,
                            void *const hwaddr)
{
    struct ifreq        iface;
    struct sockaddr_ll  addr;
    int                 socketfd, result;
    int                 ifindex = 0;

    if (!interface || !*interface) {
        errno = EINVAL;
        return -1;
    }

    socketfd = socket(AF_PACKET, SOCK_RAW, htons(protocol));
    if (socketfd == -1)
        return -1;

    do {

        memset(&iface, 0, sizeof iface);
        strncpy((char *)&iface.ifr_name, interface, IFNAMSIZ);
        result = ioctl(socketfd, SIOCGIFINDEX, &iface);
        if (result == -1)
            break;
        ifindex = iface.ifr_ifindex;

        memset(&iface, 0, sizeof iface);
        strncpy((char *)&iface.ifr_name, interface, IFNAMSIZ);
        result = ioctl(socketfd, SIOCGIFFLAGS, &iface);
        if (result == -1)
            break;
        iface.ifr_flags |= IFF_PROMISC;
        result = ioctl(socketfd, SIOCSIFFLAGS, &iface);
        if (result == -1)
            break;

        memset(&iface, 0, sizeof iface);
        strncpy((char *)&iface.ifr_name, interface, IFNAMSIZ);
        result = ioctl(socketfd, SIOCGIFHWADDR, &iface);
        if (result == -1)
            break;

        memset(&addr, 0, sizeof addr);
        addr.sll_family = AF_PACKET;
        addr.sll_protocol = htons(protocol);
        addr.sll_ifindex = ifindex;
        addr.sll_hatype = 0;
        addr.sll_pkttype = 0;
        addr.sll_halen = ETH_ALEN; /* Assume ethernet! */
        memcpy(&addr.sll_addr, &iface.ifr_hwaddr.sa_data, addr.sll_halen);
        if (hwaddr)
            memcpy(hwaddr, &iface.ifr_hwaddr.sa_data, ETH_ALEN);

        if (bind(socketfd, (struct sockaddr *)&addr, sizeof addr))
            break;

        errno = 0;
        return socketfd;

    } while (0);

    {
        const int saved_errno = errno;
        close(socketfd);
        errno = saved_errno;
        return -1;
    }
}

static unsigned int tci(const unsigned int priority,
                        const unsigned int drop,
                        const unsigned int vlan)
{
    return (vlan & 0xFFFU)
         | ((!!drop) << 12U)
         | ((priority & 7U) << 13U);
}

static size_t rawpacket_qinq(unsigned char *const buffer, size_t const length,
                             const unsigned char *const srcaddr,
                             const unsigned char *const dstaddr,
                             const unsigned int service_tci,
                             const unsigned int customer_tci,
                             const unsigned int ethertype)
{
    unsigned char *ptr = buffer;
    uint32_t       tag;
    uint16_t       type;

    if (length < 2 * ETH_ALEN + 4 + 4 + 2) {
        errno = ENOSPC;
        return (size_t)0;
    }

    memcpy(ptr, dstaddr, ETH_ALEN);
    ptr += ETH_ALEN;

    memcpy(ptr, srcaddr, ETH_ALEN);
    ptr += ETH_ALEN;

    /* Service 802.1AD tag. */
    tag = htonl( ((uint32_t)(ETH_P_8021AD) << 16U)
               | ((uint32_t)service_tci & 0xFFFFU) );
    memcpy(ptr, &tag, 4);
    ptr += 4;

    /* Client 802.1Q tag. */
    tag = htonl( ((uint32_t)(ETH_P_8021Q) << 16U)
               | ((uint32_t)customer_tci & 0xFFFFU) );
    memcpy(ptr, &tag, 4);
    ptr += 4;

    /* Ethertype tag. */
    type = htons((uint16_t)ethertype);
    memcpy(ptr, &type, 2);
    ptr += 2;

    return (size_t)(ptr - buffer);
}

#endif /* RAWPACKET_H */

sender.c:

#include <string.h>
#include <errno.h>
#include <stdio.h>
#include "rawpacket.h"

static size_t parse_data(unsigned char *const data, const size_t size,
                         const char *const string)
{
    char *ends = strncpy((char *)data, string, size);
    return (size_t)(ends - (char *)data);
}


static int parse_hwaddr(const char *const string,
                        void *const hwaddr)
{
    unsigned int addr[6];
    char         dummy;

    if (sscanf(string, " %02x:%02x:%02x:%02x:%02x:%02x %c",
                       &addr[0], &addr[1], &addr[2],
                       &addr[3], &addr[4], &addr[5],
                       &dummy) == 6 ||
        sscanf(string, " %02x%02x%02x%02x%02x%02x %c",
                       &addr[0], &addr[1], &addr[2],
                       &addr[3], &addr[4], &addr[5],
                       &dummy) == 6) {
        if (hwaddr) {
            ((unsigned char *)hwaddr)[0] = addr[0];
            ((unsigned char *)hwaddr)[1] = addr[1];
            ((unsigned char *)hwaddr)[2] = addr[2];
            ((unsigned char *)hwaddr)[3] = addr[3];
            ((unsigned char *)hwaddr)[4] = addr[4];
            ((unsigned char *)hwaddr)[5] = addr[5];
        }
        return 0;
    }

    errno = EINVAL;
    return -1;
}

int main(int argc, char *argv[])
{
    unsigned char packet[ETH_FRAME_LEN + ETH_FCS_LEN];
    unsigned char srcaddr[6], dstaddr[6];
    int           socketfd;
    size_t        size, i;
    ssize_t       n;

    if (argc < 3 || argc > 4 || !strcmp(argv[1], "-h") || !strcmp(argv[1], "--help")) {
        fprintf(stderr, "\n");
        fprintf(stderr, "Usage: %s [ -h | --help ]\n", argv[0]);
        fprintf(stderr, "       %s interface hwaddr [message]\n", argv[0]);
        fprintf(stderr, "\n");
        return 1;
    }

    if (parse_hwaddr(argv[2], &dstaddr)) {
        fprintf(stderr, "%s: Invalid destination hardware address.\n", argv[2]);
        return 1;
    }

    socketfd = rawpacket_socket(ETH_P_ALL, argv[1], &srcaddr);
    if (socketfd == -1) {
        fprintf(stderr, "%s: %s.\n", argv[1], strerror(errno));
        return 1;
    }

    memset(packet, 0, sizeof packet);

    /* Construct a QinQ header for a fake Ethernet packet type. */
    size = rawpacket_qinq(packet, sizeof packet, srcaddr, dstaddr,
                                  tci(7, 0, 1U), tci(7, 0, 2U),
                                  ETH_P_IP);
    if (!size) {
        fprintf(stderr, "Failed to construct QinQ headers: %s.\n", strerror(errno));
        close(socketfd);
        return 1;
    }

    /* Add packet payload. */
    if (argc > 3)
        size += parse_data(packet + size, sizeof packet - size, argv[3]);
    else
        size += parse_data(packet + size, sizeof packet - size, "Hello!");

    /* Pad with zeroes to minimum 64 octet length. */
    if (size < 64)
        size = 64;

    /* Send it. */
    n = send(socketfd, packet, size, 0);
    if (n == -1) {
        fprintf(stderr, "Failed to send packet: %s.\n", strerror(errno));
        shutdown(socketfd, SHUT_RDWR);
        close(socketfd);
        return 1;
    }

    fprintf(stderr, "Sent %ld bytes:", (long)n);
    for (i = 0; i < size; i++)
        fprintf(stderr, " %02x", packet[i]);
    fprintf(stderr, "\n");
    fflush(stderr);

    shutdown(socketfd, SHUT_RDWR);
    if (close(socketfd)) {
        fprintf(stderr, "Error closing socket: %s.\n", strerror(errno));
        return 1;
    }

    return 0;
}

receiver.c:

#include <sys/types.h>
#include <sys/socket.h>
#include <string.h>
#include <signal.h>
#include <errno.h>
#include <stdio.h>
#include "rawpacket.h"

static volatile sig_atomic_t  done = 0;

static void handle_done(int signum)
{
    done = signum;
}

static int install_done(const int signum)
{
    struct sigaction act;
    sigemptyset(&act.sa_mask);
    act.sa_handler = handle_done;
    act.sa_flags = 0;
    if (sigaction(signum, &act, NULL))
        return errno;
    return 0;
}

static const char *protocol_name(const unsigned int protocol)
{
    static char buffer[16];
    switch (protocol & 0xFFFFU) {
    case 0x0001: return "ETH_P_802_3";
    case 0x0002: return "ETH_P_AX25";
    case 0x0003: return "ETH_P_ALL";
    case 0x0060: return "ETH_P_LOOP";
    case 0x0800: return "ETH_P_IP";
    case 0x0806: return "ETH_P_ARP";
    case 0x8100: return "ETH_P_8021Q (802.1Q VLAN)";
    case 0x88A8: return "ETH_P_8021AD (802.1AD VLAN)";
    default:
        snprintf(buffer, sizeof buffer, "0x%04x", protocol & 0xFFFFU);
        return (const char *)buffer;
    }
}

static const char *header_type(const unsigned int hatype)
{
    static char buffer[16];
    switch (hatype) {
    case   1: return "ARPHRD_ETHER: Ethernet 10Mbps";
    case   2: return "ARPHRD_EETHER: Experimental Ethernet";
    case 768: return "ARPHRD_TUNNEL: IP Tunnel";
    case 772: return "ARPHRD_LOOP: Loopback";
    default:
        snprintf(buffer, sizeof buffer, "0x%04x", hatype);
        return buffer;
    }
}

static const char *packet_type(const unsigned int pkttype)
{
    static char buffer[16];
    switch (pkttype) {
    case PACKET_HOST:      return "PACKET_HOST";
    case PACKET_BROADCAST: return "PACKET_BROADCAST";
    case PACKET_MULTICAST: return "PACKET_MULTICAST";
    case PACKET_OTHERHOST: return "PACKET_OTHERHOST";
    case PACKET_OUTGOING:  return "PACKET_OUTGOING";
    default:
        snprintf(buffer, sizeof buffer, "0x%02x", pkttype);
        return (const char *)buffer;
    }
}

static void fhex(FILE *const out,
                 const char *const before,
                 const char *const after,
                 const void *const src, const size_t len)
{
    const unsigned char *const data = src;
    size_t i;

    if (len < 1)
        return;

    if (before)
        fputs(before, out);

    for (i = 0; i < len; i++)
        fprintf(out, " %02x", data[i]);

    if (after)
        fputs(after, out);
}

int main(int argc, char *argv[])
{
    struct sockaddr_ll  addr;
    socklen_t           addrlen;
    unsigned char       data[2048];
    ssize_t             n;
    int                 socketfd, flag;

    if (argc != 2 || !strcmp(argv[1], "-h") || !strcmp(argv[1], "--help")) {
        fprintf(stderr, "\n");
        fprintf(stderr, "Usage: %s [ -h | --help ]\n", argv[0]);
        fprintf(stderr, "       %s interface\n", argv[0]);
        fprintf(stderr, "\n");
        return 1;
    }

    if (install_done(SIGINT) ||
        install_done(SIGHUP) ||
        install_done(SIGTERM)) {
        fprintf(stderr, "Cannot install signal handlers: %s.\n", strerror(errno));
        return 1;
    }

    socketfd = rawpacket_socket(ETH_P_ALL, argv[1], NULL);
    if (socketfd == -1) {
        fprintf(stderr, "%s: %s.\n", argv[1], strerror(errno));
        return 1;
    }

    flag = 1;
    if (setsockopt(socketfd, SOL_SOCKET, SO_REUSEADDR, &flag, sizeof flag)) {
        fprintf(stderr, "Cannot set REUSEADDR socket option: %s.\n", strerror(errno));
        close(socketfd);
        return 1;
    }

    if (setsockopt(socketfd, SOL_SOCKET, SO_BINDTODEVICE, argv[1], strlen(argv[1]) + 1)) {
        fprintf(stderr, "Cannot bind to device %s: %s.\n", argv[1], strerror(errno));
        close(socketfd);
        return 1;
    }

    while (!done) {

        memset(data, 0, sizeof data);
        memset(&addr, 0, sizeof addr);
        addrlen = sizeof addr;
        n = recvfrom(socketfd, &data, sizeof data, 0,
                     (struct sockaddr *)&addr, &addrlen);
        if (n == -1) {
            if (errno == EINTR)
                continue;
            fprintf(stderr, "Receive error: %s.\n", strerror(errno));
            break;
        }

        printf("Received %d bytes:\n", (int)n);
        printf("\t    Protocol: %s\n", protocol_name(htons(addr.sll_protocol)));
        printf("\t   Interface: %d\n", (int)addr.sll_ifindex);
        printf("\t Header type: %s\n", header_type(addr.sll_hatype));
        printf("\t Packet type: %s\n", packet_type(addr.sll_pkttype));
        fhex(stdout, "\t     Address:", "\n", addr.sll_addr, addr.sll_halen);
        fhex(stdout, "\t        Data:", "\n", data, n);
        printf("\n");

        fflush(stdout);
    }

    shutdown(socketfd, SHUT_RDWR);
    close(socketfd);
    return 0;
}

要编译,您可以使用

gcc -O2 receiver.c -o receiver
gcc -O2 sender.c -o sender

不带参数运行,或使用-h,以查看其中任何一个的用法.sender只发送一个数据包.receiver监听指定的接口(在混杂模式下),直到你打断它(Ctrl+ C)或发送TERM信号.

在Loopback接口上的一个虚拟终端中启动接收器:

sudo ./receiver lo

在同一台机器上的另一个虚拟终端中运行

sudo ./sender lo FF:FF:FF:FF:FF:FF '_The contents of a 64-byte Ethernet frame_'

将输出(添加换行符和缩进以便于理解)

Sent 64 bytes: ff ff ff ff ff ff
               00 00 00 00 00 00
               88 a8 e0 01
               81 00 e0 02
               08 00
               5f 54 68 65 20 63 6f 6e
               74 65 6e 74 73 20 6f 66
               20 61 20 36 34 2d 62 79
               74 65 20 45 74 68 65 72
               6e 65 74 20 66 72 61 6d
               65 5f

但是,在接收器终端中,我们看到(添加了换行符和缩进):

Received 64 bytes:
    Protocol: ETH_P_ALL
   Interface: 1
 Header type: ATPHRD_LOOP: Loopback
 Packet type: PACKET_OUTGOING
     Address: 00 00 00 00 00 00
        Data: ff ff ff ff ff ff
              00 00 00 00 00 00
              88 a8 e0 01
              81 00 e0 02
              08 00
              5f 54 68 65 20 63 6f 6e
              74 65 6e 74 73 20 6f 66
              20 61 20 36 34 2d 62 79
              74 65 20 45 74 68 65 72
              6e 65 74 20 66 72 61 6d
              65 5f

Received 60 bytes:
    Protocol: ETH_P_8021Q (802.1Q VLAN)
   Interface: 1
 Header type: ATPHRD_LOOP: Loopback
 Packet type: PACKET_MULTICAST
     Address: 00 00 00 00 00 00
        Data: ff ff ff ff ff ff
              00 00 00 00 00 00
              81 00 e0 02
              08 00
              5f 54 68 65 20 63 6f 6e
              74 65 6e 74 73 20 6f 66
              20 61 20 36 34 2d 62 79
              74 65 20 45 74 68 65 72
              6e 65 74 20 66 72 61 6d
              65 5f

第一个,PACKET_OUTGOING被捕获为传出; 它表明内核在发送数据包时没有使用任何头文件.

第二个,PACKET_MULTICAST,在它到达时被捕获.(由于以太网地址为FF:FF:FF:FF:FF:FF,因此它是一个组播数据包.)

如您所见,后一个数据包只有802.1Q VLAN头 - 客户端VLAN - 内核使用了802.1AD服务VLAN标记.

以上确认了环回接口的场景,至少.使用原始数据包接口,内核使用802.1AD VLAN标头(紧跟在收件人地址之后的标头).如果你tcpdump -i eth0在接收器旁边使用,你会发现libpcap正在将消耗的头重新插入数据包!

Loopback接口有点特殊,所以让我们使用虚拟机重做测试.我碰巧运行的是最新的Xubuntu 14.04(所有更新都安装在2014-06-28,Ubuntu 3.13.0-29-通用#53 x86_64内核).发件人硬件地址为08 00 00 00 00 02,接收者为08 00 00 00 00 01,两者连接到内部网络,没有其他人在场.

(同样,我在输出中添加换行符和缩进以使其更易于阅读.)

发件人,在虚拟机2上:

sudo ./sender eth0 08:00:00:00:00:01 '_The contents of a 64-byte Ethernet frame_'

Sent 64 bytes: 08 00 00 00 00 01
               08 00 00 00 00 02
               88 a8 e0 01
               81 00 e0 02
               08 00
               5f 54 68 65 20 63 6f 6e
               74 65 6e 74 73 20 6f 66
               20 61 20 36 34 2d 62 79
               74 65 20 45 74 68 65 72
               6e 65 74 20 66 72 61 6d
               65 5f

接收方,在虚拟机1上:

sudo ./receiver eth0

Received 60 bytes:
    Protocol: ETH_P_8021Q (802.1Q VLAN)
   Interface: 2
 Header type: ARPHRD_ETHER: Ethernet 10Mbps
 Packet type: PACKET_HOST
     Address: 08 00 00 00 00 02
        Data: 08 00 00 00 00 01
              08 00 00 00 00 02
              81 00 e0 02
              08 00
              5f 54 68 65 20 63 6f 6e
              74 65 6e 74 73 20 6f 66
              20 61 20 36 34 2d 62 79
              74 65 20 45 74 68 65 72
              6e 65 74 20 66 72 61 6d
              65 5f

如您所见,结果与环回案例的结果基本相同.特别是,接收时消耗了802.1AD服务VLAN标记.(您可以使用tcpdump或wireshark来比较收到的数据包:libpcap显然会将消耗的VLAN标记包重新插入到数据包中.)

If you have recent enough kernel (support was added in April 2013), then you can configure a 802.1AD VLAN(s) on the recipient using:

sudo modprobe 8021q

sudo ip link add link eth0 eth0.service1 type vlan proto 802.1ad id 1

Receiving on eth0 will receive all packets, but on eth0.service1 only those packets with an 802.1AD VLAN tag, with VLAN ID 1. It does not capture frames with 802.1Q VLAN tags with the same VLAN ID, meaning that you can do full routing on receive for both 802.1AD and 802.1Q VLANs.

I didn't trust just the above test, myself; I created a number of 802.1AD and 802.1Q VLANs, with separate receive instances on each one, and changed the packet headers (not only service (first) tci() and client (second) tci() in the rawpacket_qinq() call in sender.c to change the service and client VLAN IDs, but also changing rawpacket.h, to verify that 802.1AD (88a8) and 802.1Q (8100) VLAN headers are routed correctly on receive). It all worked beautifully, without any hiccups.

In summary:

Given a recent enough Linux kernel version, Ethernet frames are correctly routed by the Linux kernel (by the 8021q module) on receive, including separate VLAN interfaces for 802.1AD and 802.1Q with the same VLAN IDs. The kernel does consume the VLAN header used for routing, even if no VLANs are configured.

Questions?