为什么 1GBit 卡的输出限制为 80 MiB？

Question

我正在尝试利用我的 1GiB 网卡提供的最大带宽，但它始终限制为 80MiB（实际兆字节）。原因是什么？卡片描述（lshw 输出）：

   description: Ethernet interface
    product: DGE-530T Gigabit Ethernet Adapter (rev 11)
    vendor: D-Link System Inc
    physical id: 0
    bus info: pci@0000:03:00.0
    logical name: eth1
    version: 11
    serial: 00:22:b0:68:70:41
    size: 1GB/s
    capacity: 1GB/s
    width: 32 bits
    clock: 66MHz
    capabilities: pm vpd bus_master cap_list rom ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt 1000bt-fd autonegotiation

该卡放置在以下 PCI 插槽中：

*-pci:2
     description: PCI bridge
     product: 82801 PCI Bridge
     vendor: Intel Corporation
     physical id: 1e
     bus info: pci@0000:00:1e.0
     version: 92
     width: 32 bits
     clock: 33MHz
     capabilities: pci subtractive_decode bus_master cap_list

PCI 不是任何 PCI Express 对吗？这是一个传统的 PCI 插槽？所以也许这就是原因？

操作系统是linux。

Answer 1

80 MB/秒其实还不错！这大约是 640mbps，非常接近 NIC 的千兆容量。如果您考虑 TCPIP 开销和磁盘速度，您可能处于最大速度。

Answer 2

尝试将其放入您的 /etc/sysctl.conf

# General 10gigabit/LFP tuning
net.core.rmem_max=16777216
net.core.wmem_max=16777216
net.ipv4.tcp_rmem=4096 87380 16777216
net.ipv4.tcp_wmem=4096 65536 16777216
net.ipv4.tcp_syncookies=1
net.ipv4.tcp_max_orphans=1048576
net.ipv4.tcp_orphan_retries=2

# Removes some internal buffering
net.ipv4.tcp_low_latency=1

# Time-wait sockets
# Do not turn on unless you know what you are doing!
#net.ipv4.tcp_tw_recycle=1
#net.ipv4.tcp_tw_reuse=1

# If PMTUD ICMP blackhole appears use
# RFC 4821, Packetization Layer Path MTU Discovery
net.ipv4.tcp_mtu_probing=1

# Netfilter's conntrack
# NB! For high-performance concerns you probably don't want to use `--state` rules at all 
#net.ipv4.netfilter.ip_conntrack_max=1048576
#net.nf_conntrack_max=1048576

# SACKs are an optimization to TCP which in normal scenarios improves considerably performance. 
# In Gigabit networks with no traffic competition these have the opposite effect. 
# To improve performance they should be turned off with: 
#net.ipv4.tcp_sack=0 

# Decrease the time default value for tcp_fin_timeout connection
net.ipv4.tcp_fin_timeout=15
# Decrease the time default value for tcp_keepalive_time connection
net.ipv4.tcp_keepalive_time=1800

# Increased backlog (default: 100/1000 depending on kernel)
net.core.netdev_max_backlog=10000
net.core.somaxconn=10000

# Timestamps adds additional 12 bytes to header and uses CPU
# NB! It caused massive problems for me under benchmark load
# with a high count of concurrent connections.
# ( http://redmine.lighttpd.net/wiki/1/Docs:Performance )
#net.ipv4.tcp_timestamps=0

# Portrange for outgoing connections
# (increase the ephemeral port range)
# NB! After that tuning you probably do not want to listen on port >= 1024
net.ipv4.ip_local_port_range=1024 65535

# Fixing 'Too many open files', Second useful on nginx+aio workloads
fs.file-max=16777216
fs.aio-max-nr=65536

# If you are under DDoS you can
kernel.panic=10
# Lower following values
#net.ipv4.tcp_synack_retries=2
#net.ipv4.tcp_syn_retries=2
#net.ipv4.netfilter.ip_conntrack_tcp_timeout_fin_wait=15
#net.ipv4.netfilter.ip_conntrack_tcp_timeout_close_wait=15
# If you under ping flood
#net.ipv4.icmp_echo_ignore_all=1

我们建立的每个连接都需要一个临时端口，因此需要一个文件描述符，默认情况下限制为 1024。为避免打开文件过多的问题，您需要修改 shell 的 ulimit。这可以在 中更改/etc/security/limits.conf，但需要注销/登录。现在，您只需 sudo 并修改当前 shell（如果您不想以 root 身份运行，则在调用 ulimit 后将 su 返回到您的非私有用户）：

ulimit -n 999999

您可以尝试的另一件事可能有助于提高 TCP 吞吐量，那就是增加接口队列的大小。为此，请执行以下操作：

ifconfig eth0 txqueuelen 1000

你可以玩拥塞控制：

sysctl net.ipv4.tcp_available_congestion_control
sysctl net.ipv4.tcp_congestion_control=htcp

还有一些低级调整，例如内核模块参数

# /sbin/modinfo e1000
..snip...
parm:           TxDescriptors:Number of transmit descriptors (array of int)
parm:           TxDescPower:Binary exponential size (2^X) of each transmit descriptor (array of int)
parm:           RxDescriptors:Number of receive descriptors (array of int)
parm:           Speed:Speed setting (array of int)
parm:           Duplex:Duplex setting (array of int)
parm:           AutoNeg:Advertised auto-negotiation setting (array of int)
parm:           FlowControl:Flow Control setting (array of int)
parm:           XsumRX:Disable or enable Receive Checksum offload (array of int)
parm:           TxIntDelay:Transmit Interrupt Delay (array of int)
parm:           TxAbsIntDelay:Transmit Absolute Interrupt Delay (array of int)
parm:           RxIntDelay:Receive Interrupt Delay (array of int)
parm:           RxAbsIntDelay:Receive Absolute Interrupt Delay (array of int)
parm:           InterruptThrottleRate:Interrupt Throttling Rate (array of int)
parm:           SmartPowerDownEnable:Enable PHY smart power down (array of int)
parm:           KumeranLockLoss:Enable Kumeran lock loss workaround (array of int)
parm:           copybreak:Maximum size of packet that is copied to a new buffer on receive 

甚至可以通过ethtool(1).

附注。阅读内核文档，尤其是Documentation/networking/scaling.txt

聚苯乙烯。在调整 TCP 性能时，您可能需要参考RFC6349

购买力平价。D-Link 不是最好的网络硬件。使用 pci-x 或 pci-64 尝试 Intel 硬件

Answer 3

您的 32 位、33Mhz PCI 总线最多可以传输 1,067 兆位每秒 (Mbps) 或 133.33 兆字节每秒 (MBps)。

千兆以太网可以每秒传输 116 兆字节 (MBps)。

因此，尽管您的卡应该能够使线路完全饱和，但由于各种开销，您实际上只能获得大约 90% 的利用率。

无论哪种方式，如果您获得每秒 80 兆字节 (MBps)，那么您就离得不远了，我现在对此感到相当满意。