PACKET_MMAP TX_RING后续send()调用的问题

Dau*_*uie 6 c sockets mmap raw-sockets linux-kernel

感谢您抽出宝贵时间来看看这个.这是我已经处理了一个星期左右的问题,而且我傻眼了.任何输入都会很棒.

我目前使用PACKET_MMAP TX_RING V3(我试过V2和V1,但我与他们有同样的问题.我已经决定使用V3,因为我能够保持一个块多ETH帧,而不是只能够在V1和V2中每个tp_block有1帧,以避免在我的端口扫描器的发送过程中进行过多的系统调用.我没有设置RX_RING,因此这不是图片的一部分.

现在我只用一台主机测试这个,我只打了5个端口,我知道它们是活跃的.

所以...这是情况:

我有一个TX_R​​ING:[1块| 块大小4096 | 32帧| 帧大小128]

TX_RING设置的代码:

int                     prepare_packetmmap_tx_ring(t_thread *thread)
{
    int                 tpacket_v;
    struct sockaddr_ll  sll_loc;

    /* Step 1 Create PF_PACKET socket.*/
    thread->sock = socket(PF_PACKET, SOCK_RAW, htons(ETH_P_ALL));
    if (thread->sock == -1)
    {
        toggle_thread_alive(thread);
        return (hermes_error(FAILURE, "socket() %s", strerror(errno)));
    }

    memset(&sll_loc, 0, sizeof(struct sockaddr_ll));
    memcpy(&sll_loc.sll_addr, thread->pool->iface.if_hwaddr, ETH_ALEN);
    sll_loc.sll_ifindex = thread->pool->iface.inx;
    sll_loc.sll_family = AF_PACKET;
    sll_loc.sll_protocol = htons(ETH_P_ALL);
    /* Bind our socket to the interface */
    if (bind(thread->sock, (sockaddr *)&sll_loc, sizeof(struct sockaddr_ll)) == -1)
    {
        toggle_thread_alive(thread);
        return (hermes_error(FAILURE, "bind() %s", strerror(errno)));
    }

    tpacket_v = TPACKET_V3;
    if (setsockopt(thread->sock, SOL_PACKET, PACKET_VERSION, &tpacket_v, sizeof(tpacket_v)) < 0)
    {
        toggle_thread_alive(thread);
        return (hermes_error(FAILURE, "setsockopt() PACKET_VERSION %s", strerror(errno)));
    }

    /* Step 3 determine sizes for PACKET_TX_RING and allocate txring via setsockopt() */
    thread->txring.tpr.tp_block_size = (uint)getpagesize();
    thread->txring.tpr.tp_frame_size = TPACKET3_HDRLEN + sizeof(struct ethhdr) + sizeof(struct iphdr) + DEF_TRAN_HDRLEN + thread->pool->env->cpayload_len;
    thread->txring.tpr.tp_frame_size = pow2_round(thread->txring.tpr.tp_frame_size);
    thread->txring.tpr.tp_block_nr = (THRD_HSTGRP_MAX / (thread->txring.tpr.tp_block_size / thread->txring.tpr.tp_frame_size));
    thread->txring.tpr.tp_frame_nr = thread->txring.tpr.tp_block_nr * (thread->txring.tpr.tp_block_size / thread->txring.tpr.tp_frame_size);
    thread->txring.size = thread->txring.tpr.tp_block_size * thread->txring.tpr.tp_block_nr;
    thread->txring.doffset = sizeof(struct tpacket3_hdr);

    if (setsockopt(thread->sock, SOL_PACKET, PACKET_TX_RING, (void *)&thread->txring.tpr, sizeof(thread->txring.tpr)) < 0)
    {
        toggle_thread_alive(thread);
        return (hermes_error(FAILURE, "setsockopt() PACKET_TX_RING %s", strerror(errno)));
    }

    printf("frame size: %d | blocksize: %d | block count %d | frame count %d\n", thread->txring.tpr.tp_frame_size, thread->txring.tpr.tp_block_size, thread->txring.tpr.tp_block_nr, thread->txring.tpr.tp_frame_nr);
    extra_sock_opts(thread->sock, thread->txring.size);
    /* Step 4 actually map ring to user space */
    if (!(thread->txring.ring = mmap(0, thread->txring.size,
            PROT_READ | PROT_WRITE,
            MAP_SHARED,
            thread->sock, 0)))
    {
        toggle_thread_alive(thread);
        return (hermes_error(FAILURE, "mmap() TX_RING %s", strerror(errno)));
    }
    return (SUCCESS);
}
Run Code Online (Sandbox Code Playgroud)

所以...一切都很好,花花公子吧?让我们继续我的戒指填充过程:

        nt                      fill_tx_ring(t_thread *thread, t_frame *ethframe)
{
    struct tpacket3_hdr *frame;
    void                *data;
    uint16_t            *srcports;
    uint16_t            *dstports;
    uint                ring_i;
    uint                hst_i;
    int                 ret;

    ring_i = 0;
    hst_i = 0;
    ret = 0;
    srcports = thread->pool->env->ports.flat;
    dstports = thread->pool->env->dstports;
    while (hst_i < thread->hstgrpsz && ring_i < thread->txring.tpr.tp_frame_nr)
    {
        if (thread->hstgrp[hst_i].health.done == true)
        {
            hst_i++;
            continue;
        }
        frame = (struct tpacket3_hdr *)(thread->txring.ring + (thread->txring.tpr.tp_frame_size * ring_i));
        switch((volatile uint32_t)frame->tp_status)
        {
            case TP_STATUS_WRONG_FORMAT:
                return(hermes_error(FAILURE, "TX_RING wrong format in frame %i of thread %d", ring_i, thread->id));
            case TP_STATUS_AVAILABLE:
                data = (uint8_t *)frame + thread->txring.doffset;
                ethframe->ip->daddr = thread->hstgrp[hst_i].result->ip.s_addr;
                ip_checksum(ethframe->ip);
                ethframe->tcp->source = htons(dstports[thread->hstgrp[hst_i].health.portinx]);
                ethframe->tcp->dest = htons(srcports[thread->hstgrp[hst_i].health.portinx]);
                tcp_checksum(ethframe->ip, (uint16_t *)ethframe->tcp);
                memcpy(data, ethframe->buffer, ethframe->size);
                frame->tp_next_offset = 0;
                frame->tp_len = ethframe->size;
                frame->tp_status = TP_STATUS_SEND_REQUEST;
                printf("setup a frame\n");
                ret++; hst_i++; ring_i++;
                break;
            default:
                printf("skipped ring with status %d\n", frame->tp_status);
                ring_i++;
                break;
        }
    }
    return (ret);
}
Run Code Online (Sandbox Code Playgroud)

(编辑/注意:我也尝试在我的套接字上轮询POLL_OUT,直到tp_status在一个帧上发生变化,这会产生相同的结果.)

因此,这个填充过程将填满一帧,因为我只扫描一个主机,而我一次只向主机发送一条消息.第一次发送很棒!它发送框架,我在Wireshark上看到它,我得到响应,我处理它,然后我们回到fill_tx_ring()例程.下面是我的扫描功能,因此您可以看到发生这种情况的顺序:

void                    syn_scan(t_thread *thread)
{
    struct timeval      sent = {0};
    struct timeval      now = {0};
    long ms;
    int                 ret;
    t_frame             frame;

    memset(&frame, 0, sizeof(t_frame));
    init_ethframe(thread, &frame);
    thread->scancnt = thread->hstgrpsz;
    while (thread->scancnt > 0)
    {
        gettimeofday(&now, NULL);
        if ((ms = timediff_ms(&now, &sent)) >= DEF_INIT_RTT_TIMEOUT)
        {
            fill_tx_ring(thread, &frame);
            send_task(thread);
            gettimeofday(&sent, NULL);
            thread->rxfilter.fd.events = POLL_IN;
            if ((ret = poll(&thread->rxfilter.fd, 1, 
DEF_INIT_RTT_TIMEOUT)) < 0)
                hermes_error(FAILURE, "poll() %s", strerror(errno));
            if (ret > 0)
                pcap_dispatch(thread->rxfilter.handle,
                              thread->hstgrpsz, handle_packet, 
(u_char*)thread);
            else
                printf("didn't get anything\n");
        }
        else
        {
            usleep((useconds_t) ms * 1000);
        }

    }
    free(frame.buffer);
    frame.buffer = NULL;
}
Run Code Online (Sandbox Code Playgroud)

现在真正的问题:所有其他调用send()返回0.在随后的调用来fill_tx_ring(),我发现,所有因未能发送帧我填补以前不被内核(更新中()似乎内核甚至从未收到请求或查看环)...他们的tp_status字段仍设置为TP_STATUS_SEND_REQUEST.

即使我们有一个成功的send()(比方说,第三次成功),你会发现,它已经被用于发送帧帧都仍设置为1(TP_STATUS_SEND_REQUEST).

我程序的记录输出如下所示:

index 2 | mac: a8:7d:12:1:b6:46:| ip: 10.43.1.162 | gwip: 10.43.1.254 
| gwmac: 0:35:1a:54:ca:17:
frame size: 128 | blocksize: 4096 | block count 1 | frame count 32
ports total 5

setup a frame
sent 1 packets
packet processed
a8  7d 12 01 b6 46 00 35 
1a 54 ca 17 08 00 45  00 00 2c 00 00 40 00 
3f 06 1f 7c 0a 72 06  12 0a 2b 01 a2 16 00 
b1 f8 52 03 f6 a0 00  00 00 02 60 12 ff ff 
81 11 00 00 02 04 05  b4 00 00 

setup a frame
didn't send anything.
didn't get anything

skipped ring with status 1
setup a frame
sent 1 packets
packet processed
a8  7d 12 01 b6 46 00 35 
1a 54 ca 17 08 00 45  00 00 2c 00 00 40 00 
3f 06 1f 7c 0a 72 06  12 0a 2b 01 a2 58 00 
ac ac 06 32 5f 44 00  00 00 02 60 12 ff ff 
69 49 00 00 02 04 05  b4 00 00 

skipped ring with status 1
setup a frame
didn't send anything.
didn't get anything

skipped ring with status 1
skipped ring with status 1
setup a frame
sent 1 packets
packet processed
a8  7d 12 01 b6 46 00 35 
1a 54 ca 17 08 00 45  00 00 2c 00 00 40 00 
3f 06 1f 7c 0a 72 06  12 0a 2b 01 a2 d3 0c 
e7 f1 14 3c 13 c4 00  00 00 02 60 12 ff ff 
5e ff 00 00 02 04 05  b4 00 00 

skipped ring with status 1
skipped ring with status 1
setup a frame
didn't send anything.
didn't get anything

skipped ring with status 1
skipped ring with status 1
skipped ring with status 1
setup a frame
sent 1 packets
packet processed
a8  7d 12 01 b6 46 00 35 
1a 54 ca 17 08 00 45  00 00 2c 00 00 40 00 
3f 06 1f 7c 0a 72 06  12 0a 2b 01 a2 ea 0c 
8a 10 f0 d0 aa 9b 00  00 00 02 60 12 ff ff 
49 5d 00 00 02 04 05  b4 00 00 

skipped ring with status 1
skipped ring with status 1
skipped ring with status 1
setup a frame
didn't send anything.
didn't get anything
ms 1000
skipped ring with status 1
skipped ring with status 1
skipped ring with status 1
skipped ring with status 1
setup a frame
sent 1 packets
packet processed
a8  7d 12 01 b6 46 00 35 
1a 54 ca 17 08 00 45  00 00 2c 00 00 40 00 
3f 06 1f 7c 0a 72 06  12 0a 2b 01 a2 0c 17 
ee 5f a7 98 89 b2 00  00 00 02 60 12 ff ff 
45 0d 00 00 02 04 05  b4 00 00 
thread 1 closing 0 still alive
Run Code Online (Sandbox Code Playgroud)

那么为什么内核没有给我回复帧,为什么每次调用send()都无法发送我在环中设置的帧?

提前多多欣赏!

你可以看到完整的代码:

项目:https: //github.com/dauie/hermes.git

tx setup:https: //github.com/Dauie/hermes/blob/master/src/thread.c

tx_fill/send:https: //github.com/Dauie/hermes/blob/master/src/scan.c