Strange dropped packets results in TC
From: Lucas Bocchi <hidden>
Date: 2021-06-15 12:55:07
Hello Folks We have a little doubt about dropped packets in tc util. As you see, my interface doesn't show any dropped packets in ethtool or in kernel statistics, but in tc is showed dropped packets. And that ocurrs in two different servers, with two different network interface / queue disciplines. The kernel Server 1 - Intel IXGBE Driver with Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Card and TP-LINK TL-SM321B Transceiver Transceiver Statistics: ethtool -m enp1s0f0 Identifier : 0x03 (SFP) Extended identifier : 0x04 (GBIC/SFP defined by 2-wire interface ID) Connector : 0x07 (LC) Transceiver codes : 0x00 0x00 0x00 0x02 0x82 0x00 0x01 0x01 0x00 Transceiver type : Ethernet: 1000BASE-LX Transceiver type : FC: very long distance (V) Transceiver type : FC: Longwave laser (LC) Transceiver type : FC: Single Mode (SM) Transceiver type : FC: 100 MBytes/sec Encoding : 0x01 (8B/10B) BR, Nominal : 1300MBd Rate identifier : 0x00 (unspecified) Length (SMF,km) : 20km Length (SMF) : 20000m Length (50um) : 0m Length (62.5um) : 0m Length (Copper) : 0m Length (OM3) : 0m Laser wavelength : 1310nm Vendor name : TP-LINK Vendor OUI : 00:00:00 Vendor PN : TL-SM321B Vendor rev : 1.1 Option values : 0x00 0x1a Option : RX_LOS implemented Option : TX_FAULT implemented Option : TX_DISABLE implemented BR margin, max : 0% BR margin, min : 0% Vendor SN : 35704205710058 Date code : 170517 Ethtool Statistic NIC statistics: rx_packets: 1830508376 tx_packets: 2095191306 rx_bytes: 1559661300658 tx_bytes: 1796376456250 rx_pkts_nic: 1830508375 tx_pkts_nic: 2095191306 rx_bytes_nic: 1566983334162 tx_bytes_nic: 1806513286636 lsc_int: 1 tx_busy: 0 non_eop_descs: 0 rx_errors: 0 tx_errors: 0 // No TX Error rx_dropped: 55706 tx_dropped: 0 // No dropped Packets multicast: 55706 broadcast: 45 rx_no_buffer_count: 0 collisions: 0 rx_over_errors: 0 rx_crc_errors: 0 rx_frame_errors: 0 hw_rsc_aggregated: 0 hw_rsc_flushed: 0 fdir_match: 1335933692 fdir_miss: 548533583 fdir_overflow: 888 rx_fifo_errors: 0 rx_missed_errors: 0 tx_aborted_errors: 0 tx_carrier_errors: 0 tx_fifo_errors: 0 tx_heartbeat_errors: 0 tx_timeout_count: 0 tx_restart_queue: 0 rx_length_errors: 0 rx_long_length_errors: 0 rx_short_length_errors: 0 tx_flow_control_xon: 0 rx_flow_control_xon: 0 tx_flow_control_xoff: 0 rx_flow_control_xoff: 0 rx_csum_offload_errors: 87755 alloc_rx_page: 184014 alloc_rx_page_failed: 0 alloc_rx_buff_failed: 0 rx_no_dma_resources: 0 os2bmc_rx_by_bmc: 0 os2bmc_tx_by_bmc: 0 os2bmc_tx_by_host: 0 os2bmc_rx_by_host: 0 tx_hwtstamp_timeouts: 0 tx_hwtstamp_skipped: 0 rx_hwtstamp_cleared: 0 tx_ipsec: 0 rx_ipsec: 0 fcoe_bad_fccrc: 0 rx_fcoe_dropped: 0 rx_fcoe_packets: 0 rx_fcoe_dwords: 0 fcoe_noddp: 0 fcoe_noddp_ext_buff: 0 tx_fcoe_packets: 0 tx_fcoe_dwords: 0 tx_queue_0_packets: 293242265 tx_queue_0_bytes: 247153468767 tx_queue_1_packets: 288898419 tx_queue_1_bytes: 245035641991 tx_queue_2_packets: 291236572 tx_queue_2_bytes: 249561212685 tx_queue_3_packets: 39837597 tx_queue_3_bytes: 2358074882 tx_queue_4_packets: 307730121 tx_queue_4_bytes: 264761824523 tx_queue_5_packets: 317352090 tx_queue_5_bytes: 277380973829 tx_queue_6_packets: 300122596 tx_queue_6_bytes: 258114228358 tx_queue_7_packets: 256771646 tx_queue_7_bytes: 252011031215 ... rx_queue_0_packets: 251550997 rx_queue_0_bytes: 212000117060 rx_queue_1_packets: 253297759 rx_queue_1_bytes: 210961040647 rx_queue_2_packets: 254114246 rx_queue_2_bytes: 218970216150 rx_queue_3_packets: 71013743 rx_queue_3_bytes: 23381820239 rx_queue_4_packets: 267448221 rx_queue_4_bytes: 236201977962 rx_queue_5_packets: 281248119 rx_queue_5_bytes: 232497403664 rx_queue_6_packets: 262045046 rx_queue_6_bytes: 223232203577 rx_queue_7_packets: 189790245 rx_queue_7_bytes: 202416521359 ... ip statistics ip -s -s addr show dev enp1s0f0 4: enp1s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc multiq state UP group default qlen 1000 link/ether 90:e2:ba:89:11:70 brd ff:ff:ff:ff:ff:ff inet 187.60.215.230/30 brd 187.60.215.231 scope global enp1s0f0 valid_lft forever preferred_lft forever inet6 2804:6fc:1:10::66/126 scope global valid_lft forever preferred_lft forever inet6 fe80::92e2:baff:fe89:1170/64 scope link valid_lft forever preferred_lft forever RX: bytes packets errors dropped missed mcast 1563726136109 1835102960 0 55846 0 55845 RX errors: length crc frame fifo overrun 0 0 0 0 0 TX: bytes packets errors dropped carrier collsns 1799499580192 2099722447 0 0 0 0 // No packet dropped TX errors: aborted fifo window heartbeat transns 0 0 0 0 2 tc qdisc tc -s qdisc ls dev enp1s0f0 qdisc multiq 8003: root refcnt 65 bands 8/64 Sent 1760376270449 bytes 2048050689 pkt (dropped 69, overlimits 0 requeues 477) backlog 0b 0p requeues 477 tc -s class ls dev enp1s0f0 class multiq 8003:1 parent 8003: Sent 242641753495 bytes 286672005 pkt (dropped 15, overlimits 0 requeues 0) backlog 0b 0p requeues 0 class multiq 8003:2 parent 8003: Sent 241001820689 bytes 282960042 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 class multiq 8003:3 parent 8003: Sent 244006535406 bytes 284363659 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 class multiq 8003:4 parent 8003: Sent 2290044667 bytes 39029076 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 class multiq 8003:5 parent 8003: Sent 260775212492 bytes 301997242 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 class multiq 8003:6 parent 8003: Sent 271403512839 bytes 309429699 pkt (dropped 54, overlimits 0 requeues 0) backlog 0b 0p requeues 0 class multiq 8003:7 parent 8003: Sent 252190559164 bytes 292911989 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 class multiq 8003:8 parent 8003: Sent 245887513841 bytes 250404773 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 Server 2 - Intel Corporation Ethernet Connection (12) I219-V ethtool -S eno1 NIC statistics: rx_packets: 16327063 tx_packets: 8024473 rx_bytes: 22725903298 tx_bytes: 3014818845 rx_broadcast: 2598 tx_broadcast: 1 rx_multicast: 0 tx_multicast: 7 rx_errors: 0 tx_errors: 0 tx_dropped: 0 // No Dropped or error in TX multicast: 0 collisions: 0 rx_length_errors: 0 rx_over_errors: 0 rx_crc_errors: 0 rx_frame_errors: 0 rx_no_buffer_count: 0 rx_missed_errors: 0 tx_aborted_errors: 0 tx_carrier_errors: 0 tx_fifo_errors: 0 tx_heartbeat_errors: 0 tx_window_errors: 0 tx_abort_late_coll: 0 tx_deferred_ok: 0 tx_single_coll_ok: 0 tx_multi_coll_ok: 0 tx_timeout_count: 0 tx_restart_queue: 0 rx_long_length_errors: 0 rx_short_length_errors: 0 rx_align_errors: 0 tx_tcp_seg_good: 0 tx_tcp_seg_failed: 0 rx_flow_control_xon: 0 rx_flow_control_xoff: 0 tx_flow_control_xon: 0 tx_flow_control_xoff: 0 rx_csum_offload_good: 0 rx_csum_offload_errors: 0 rx_header_split: 0 alloc_rx_buff_failed: 0 tx_smbus: 0 rx_smbus: 0 dropped_smbus: 0 rx_dma_failed: 0 tx_dma_failed: 0 rx_hwtstamp_cleared: 0 uncorr_ecc_errors: 0 corr_ecc_errors: 0 tx_hwtstamp_timeouts: 0 tx_hwtstamp_skipped: 0 ip statistics 2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo state UP mode DEFAULT group default qlen 1000 link/ether a8:a1:59:5a:56:e7 brd ff:ff:ff:ff:ff:ff RX: bytes packets errors dropped missed mcast 22738286184 16335559 0 2600 0 0 RX errors: length crc frame fifo overrun 0 0 0 0 0 TX: bytes packets errors dropped carrier collsns 3015100201 8026007 0 0 0 0 TX errors: aborted fifo window heartbeat transns 0 0 0 0 2 altname enp0s31f6 tc statistic tc -s qdisc ls dev eno1 qdisc pfifo 8002: root refcnt 2 limit 1000p Sent 2982635243 bytes 8027086 pkt (dropped 247, overlimits 0 requeues 4254) backlog 0b 0p requeues 4254 Any tip to try to identify that problem?