From: Maciej Fijalkowski <maciej.fijalkowski@intel.com> Date: 2021-01-18 15:23:53
Hi,
This series is mostly about the cleanups on Rx (ZC/normal) paths both in
ice and i40e drivers. Things that stand out are the simplifactions of
ice_change_mtu and i40e_xdp_setup.
Third iteration of this includes patches that optimize the handling of
*_rx_offset() calls per each processed frame. Some cycles can be saved
by storing the result of that function onto rx ring. For that, I am
using existing holes within ring structs (checked with pahole).
Thanks!
v3: rebase, fix handling rx offset
v2: fix kdoc in patch 5 (Jakub)
Björn Töpel (1):
i40e, xsk: Simplify the do-while allocation loop
Maciej Fijalkowski (10):
i40e: drop redundant check when setting xdp prog
i40e: drop misleading function comments
i40e: adjust i40e_is_non_eop
ice: simplify ice_run_xdp
ice: move skb pointer from rx_buf to rx_ring
ice: remove redundant checks in ice_change_mtu
ice: skip NULL check against XDP prog in ZC path
i40e: store the result of i40e_rx_offset() onto i40e_ring
ice: store the result of ice_rx_offset() onto ice_ring
ixgbe: store the result of ixgbe_rx_offset() onto ixgbe_ring
drivers/net/ethernet/intel/i40e/i40e_main.c | 3 -
drivers/net/ethernet/intel/i40e/i40e_txrx.c | 91 ++++++-------------
drivers/net/ethernet/intel/i40e/i40e_txrx.h | 1 +
drivers/net/ethernet/intel/i40e/i40e_xsk.c | 4 +-
drivers/net/ethernet/intel/ice/ice_main.c | 9 --
drivers/net/ethernet/intel/ice/ice_txrx.c | 88 ++++++++----------
drivers/net/ethernet/intel/ice/ice_txrx.h | 3 +-
drivers/net/ethernet/intel/ice/ice_xsk.c | 7 +-
drivers/net/ethernet/intel/ixgbe/ixgbe.h | 1 +
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 15 +--
10 files changed, 86 insertions(+), 136 deletions(-)
--
2.20.1
From: Maciej Fijalkowski <maciej.fijalkowski@intel.com> Date: 2021-01-18 15:24:05
Net core handles the case where netdev has no xdp prog attached and
current prog is NULL. Therefore, remove such check within
i40e_xdp_setup.
Reviewed-by: Björn Töpel <redacted>
Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
drivers/net/ethernet/intel/i40e/i40e_main.c | 3 ---
1 file changed, 3 deletions(-)
@@ -12462,9 +12462,6 @@ static int i40e_xdp_setup(struct i40e_vsi *vsi,if(frame_size>vsi->rx_buf_len)return-EINVAL;-if(!i40e_enabled_xdp_vsi(vsi)&&!prog)-return0;-/* When turning XDP on->off/off->on we reset and rebuild the rings. */need_reset=(i40e_enabled_xdp_vsi(vsi)!=!!prog);
From: Maciej Fijalkowski <maciej.fijalkowski@intel.com> Date: 2021-01-18 15:24:12
i40e_cleanup_headers has a statement about check against skb being
linear or not which is not relevant anymore, so let's remove it.
Same case for i40e_can_reuse_rx_page, it references things that are not
present there anymore.
Reviewed-by: Björn Töpel <redacted>
Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
drivers/net/ethernet/intel/i40e/i40e_txrx.c | 33 ++++-----------------
1 file changed, 6 insertions(+), 27 deletions(-)
From: Maciej Fijalkowski <maciej.fijalkowski@intel.com> Date: 2021-01-18 15:24:25
i40e_is_non_eop had a leftover comment and unused skb argument which was
used for placing the skb onto rx_buf in case when current buffer was
non-eop one. This is not relevant anymore as commit e72e56597ba1
("i40e/i40evf: Moves skb from i40e_rx_buffer to i40e_ring") pulled the
non-complete skb handling out of rx_bufs up to rx_ring. Therefore,
let's adjust the function arguments that i40e_is_non_eop takes.
Furthermore, since there is already a function responsible for bumping
the ntc, make use of that and drop that logic from i40e_is_non_eop so
that the scope of this function is limited to what the name actually
states.
Reviewed-by: Björn Töpel <redacted>
Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
drivers/net/ethernet/intel/i40e/i40e_txrx.c | 23 ++++++---------------
1 file changed, 6 insertions(+), 17 deletions(-)
@@ -2130,25 +2130,13 @@ static void i40e_put_rx_buffer(struct i40e_ring *rx_ring,*i40e_is_non_eop-processhandlingofnon-EOPbuffers*@rx_ring:Rxringbeingprocessed*@rx_desc:Rxdescriptorforcurrentbuffer-*@skb:Currentsocketbuffercontainingbufferinprogress*-*Thisfunctionupdatesnexttoclean.IfthebufferisanEOPbuffer-*thisfunctionexitsreturningfalse,otherwiseitwillplacethe-*sk_buffinthenextbuffertobechainedandreturntrueindicating-*thatthisisinfactanon-EOPbuffer.-**/+*IfthebufferisanEOPbuffer,thisfunctionexitsreturningfalse,+*otherwisereturntrueindicatingthatthisisinfactanon-EOPbuffer.+*/staticbooli40e_is_non_eop(structi40e_ring*rx_ring,-unioni40e_rx_desc*rx_desc,-structsk_buff*skb)+unioni40e_rx_desc*rx_desc){-u32ntc=rx_ring->next_to_clean+1;--/* fetch, update, and store next to clean */-ntc=(ntc<rx_ring->count)?ntc:0;-rx_ring->next_to_clean=ntc;--prefetch(I40E_RX_DESC(rx_ring,ntc));-/* if we are the last buffer then there is nothing else to do */#define I40E_RXD_EOF BIT(I40E_RX_DESC_STATUS_EOF_SHIFT)if(likely(i40e_test_staterr(rx_desc,I40E_RXD_EOF)))
@@ -2427,7 +2415,8 @@ static int i40e_clean_rx_irq(struct i40e_ring *rx_ring, int budget)i40e_put_rx_buffer(rx_ring,rx_buffer,rx_buffer_pgcnt);cleaned_count++;-if(i40e_is_non_eop(rx_ring,rx_desc,skb))+i40e_inc_ntc(rx_ring);+if(i40e_is_non_eop(rx_ring,rx_desc))continue;if(i40e_cleanup_headers(rx_ring,skb,rx_desc)){
From: Maciej Fijalkowski <maciej.fijalkowski@intel.com> Date: 2021-01-18 15:25:23
Output of ice_rx_offset() is based on ethtool's priv flag setting, which
when changed, causes PF reset (disables napi, frees irqs, loads
different Rx mem model, etc.). This means that within napi its result is
constant and there is no reason to call it per each processed frame.
Add new 'rx_offset' field to ice_ring that is meant to hold the
ice_rx_offset() result and use it within ice_clean_rx_irq().
Furthermore, use it within ice_alloc_mapped_page().
Reviewed-by: Björn Töpel <redacted>
Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
drivers/net/ethernet/intel/ice/ice_txrx.c | 43 ++++++++++++-----------
drivers/net/ethernet/intel/ice/ice_txrx.h | 1 +
2 files changed, 23 insertions(+), 21 deletions(-)
@@ -1080,6 +1081,7 @@ int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget){unsignedinttotal_rx_bytes=0,total_rx_pkts=0,frame_sz=0;u16cleaned_count=ICE_DESC_UNUSED(rx_ring);+unsignedintoffset=rx_ring->rx_offset;unsignedintxdp_res,xdp_xmit=0;structsk_buff*skb=rx_ring->skb;structbpf_prog*xdp_prog=NULL;
@@ -1094,7 +1096,6 @@ int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget)/* start the loop to process Rx packets bounded by 'budget' */while(likely(total_rx_pkts<(unsignedint)budget)){-unsignedintoffset=ice_rx_offset(rx_ring);unionice_32b_rx_flex_desc*rx_desc;structice_rx_buf*rx_buf;unsignedchar*hard_start;
From: Maciej Fijalkowski <maciej.fijalkowski@intel.com> Date: 2021-01-18 15:26:55
dev_validate_mtu checks that mtu value specified by user is not less
than min mtu and not greater than max allowed mtu. It is being done
before calling the ndo_change_mtu exposed by driver, so remove these
redundant checks in ice_change_mtu.
Reviewed-by: Björn Töpel <redacted>
Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
drivers/net/ethernet/intel/ice/ice_main.c | 9 ---------
1 file changed, 9 deletions(-)
@@ -6123,15 +6123,6 @@ static int ice_change_mtu(struct net_device *netdev, int new_mtu)}}-if(new_mtu<(int)netdev->min_mtu){-netdev_err(netdev,"new MTU invalid. min_mtu is %d\n",-netdev->min_mtu);-return-EINVAL;-}elseif(new_mtu>(int)netdev->max_mtu){-netdev_err(netdev,"new MTU invalid. max_mtu is %d\n",-netdev->min_mtu);-return-EINVAL;-}/* if a reset is in progress, wait for some time for it to complete */do{if(ice_is_reset_in_progress(pf->state)){
From: Maciej Fijalkowski <maciej.fijalkowski@intel.com> Date: 2021-01-18 15:31:36
Output of i40e_rx_offset() is based on ethtool's priv flag setting,
which when changed, causes PF reset (disables napi, frees irqs, loads
different Rx mem model, etc.). This means that within napi its result is
constant and there is no reason to call it per each processed frame.
Add new 'rx_offset' field to i40e_ring that is meant to hold the
i40e_rx_offset() result and use it within i40e_clean_rx_irq().
Furthermore, use it within i40e_alloc_mapped_page().
Last but not least, un-inline the function of interest so that compiler
makes the decision about inlining as it lives in .c file.
Reviewed-by: Björn Töpel <redacted>
Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
drivers/net/ethernet/intel/i40e/i40e_txrx.c | 35 +++++++++++----------
drivers/net/ethernet/intel/i40e/i40e_txrx.h | 1 +
2 files changed, 19 insertions(+), 17 deletions(-)
@@ -1443,6 +1454,7 @@ int i40e_setup_rx_descriptors(struct i40e_ring *rx_ring)rx_ring->next_to_alloc=0;rx_ring->next_to_clean=0;rx_ring->next_to_use=0;+rx_ring->rx_offset=i40e_rx_offset(rx_ring);/* XDP RX-queue info only needed for RX rings exposed to XDP */if(rx_ring->vsi->type==I40E_VSI_MAIN){
@@ -2373,7 +2375,6 @@ static int i40e_clean_rx_irq(struct i40e_ring *rx_ring, int budget)/* retrieve a buffer from the ring */if(!skb){-unsignedintoffset=i40e_rx_offset(rx_ring);unsignedchar*hard_start;hard_start=page_address(rx_buffer->page)+
From: Maciej Fijalkowski <maciej.fijalkowski@intel.com> Date: 2021-01-18 15:31:56
Output of ixgbe_rx_offset() is based on ethtool's priv flag setting, which
when changed, causes PF reset (disables napi, frees irqs, loads
different Rx mem model, etc.). This means that within napi its result is
constant and there is no reason to call it per each processed frame.
Add new 'rx_offset' field to ixgbe_ring that is meant to hold the
ixgbe_rx_offset() result and use it within ixgbe_clean_rx_irq().
Furthermore, use it within ixgbe_alloc_mapped_page().
Last but not least, un-inline the function of interest as it lives in .c
file so let compiler do the decision about the inlining.
Reviewed-by: Björn Töpel <redacted>
Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
drivers/net/ethernet/intel/ixgbe/ixgbe.h | 1 +
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 15 ++++++++-------
2 files changed, 9 insertions(+), 7 deletions(-)
@@ -2335,7 +2336,6 @@ static int ixgbe_clean_rx_irq(struct ixgbe_q_vector *q_vector,/* retrieve a buffer from the ring */if(!skb){-unsignedintoffset=ixgbe_rx_offset(rx_ring);unsignedchar*hard_start;hard_start=page_address(rx_buffer->page)+
@@ -6583,6 +6583,7 @@ int ixgbe_setup_rx_resources(struct ixgbe_adapter *adapter,rx_ring->next_to_clean=0;rx_ring->next_to_use=0;+rx_ring->rx_offset=ixgbe_rx_offset(rx_ring);/* XDP RX-queue info */if(xdp_rxq_info_reg(&rx_ring->xdp_rxq,adapter->netdev,
From: Maciej Fijalkowski <maciej.fijalkowski@intel.com> Date: 2021-01-18 15:56:47
There's no need for 'result' variable, we can directly return the
internal status based on action returned by xdp prog.
Reviewed-by: Björn Töpel <redacted>
Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
drivers/net/ethernet/intel/ice/ice_txrx.c | 15 +++++----------
1 file changed, 5 insertions(+), 10 deletions(-)
From: Maciej Fijalkowski <maciej.fijalkowski@intel.com> Date: 2021-01-18 15:58:09
Whole zero-copy variant of clean Rx irq is executed when xsk_pool is
attached to rx_ring and it can happen only when XDP program is present
on interface. Therefore it is safe to assume that program is always
!NULL and there is no need for checking it in ice_run_xdp_zc.
Reviewed-by: Björn Töpel <redacted>
Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
drivers/net/ethernet/intel/ice/ice_xsk.c | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
@@ -517,11 +517,10 @@ ice_run_xdp_zc(struct ice_ring *rx_ring, struct xdp_buff *xdp)u32act;rcu_read_lock();+/* ZC patch is enabled only when XDP program is set,+*sohereitcannotbeNULL+*/xdp_prog=READ_ONCE(rx_ring->xdp_prog);-if(!xdp_prog){-rcu_read_unlock();-returnICE_XDP_PASS;-}act=bpf_prog_run_xdp(xdp_prog,xdp);switch(act){
From: Maciej Fijalkowski <maciej.fijalkowski@intel.com> Date: 2021-01-18 16:41:20
Similar thing has been done in i40e, as there is no real need for having
the sk_buff pointer in each rx_buf. Non-eop frames can be simply handled
on that pointer moved upwards to rx_ring.
Reviewed-by: Björn Töpel <redacted>
Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
drivers/net/ethernet/intel/ice/ice_txrx.c | 30 ++++++++++-------------
drivers/net/ethernet/intel/ice/ice_txrx.h | 2 +-
2 files changed, 14 insertions(+), 18 deletions(-)
@@ -1042,29 +1041,24 @@ ice_put_rx_buf(struct ice_ring *rx_ring, struct ice_rx_buf *rx_buf,/* clear contents of buffer_info */rx_buf->page=NULL;-rx_buf->skb=NULL;}/***ice_is_non_eop-processhandlingofnon-EOPbuffers*@rx_ring:Rxringbeingprocessed*@rx_desc:Rxdescriptorforcurrentbuffer-*@skb:Currentsocketbuffercontainingbufferinprogress**IfthebufferisanEOPbuffer,thisfunctionexitsreturningfalse,*otherwisereturntrueindicatingthatthisisinfactanon-EOPbuffer.*/staticbool-ice_is_non_eop(structice_ring*rx_ring,unionice_32b_rx_flex_desc*rx_desc,-structsk_buff*skb)+ice_is_non_eop(structice_ring*rx_ring,unionice_32b_rx_flex_desc*rx_desc){/* if we are the last buffer then there is nothing else to do */#define ICE_RXD_EOF BIT(ICE_RX_FLEX_DESC_STATUS0_EOF_S)if(likely(ice_test_staterr(rx_desc,ICE_RXD_EOF)))returnfalse;-/* place skb in next buffer to be received */-rx_ring->rx_buf[rx_ring->next_to_clean].skb=skb;rx_ring->rx_stats.non_eop_descs++;returntrue;
@@ -1087,6 +1081,7 @@ int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget)unsignedinttotal_rx_bytes=0,total_rx_pkts=0,frame_sz=0;u16cleaned_count=ICE_DESC_UNUSED(rx_ring);unsignedintxdp_res,xdp_xmit=0;+structsk_buff*skb=rx_ring->skb;structbpf_prog*xdp_prog=NULL;structxdp_buffxdp;boolfailure;
@@ -1103,7 +1098,6 @@ int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget)unionice_32b_rx_flex_desc*rx_desc;structice_rx_buf*rx_buf;unsignedchar*hard_start;-structsk_buff*skb;unsignedintsize;u16stat_err_bits;intrx_buf_pgcnt;
@@ -1138,7 +1132,7 @@ int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget)ICE_RX_FLX_DESC_PKT_LEN_M;/* retrieve a buffer from the ring */-rx_buf=ice_get_rx_buf(rx_ring,&skb,size,&rx_buf_pgcnt);+rx_buf=ice_get_rx_buf(rx_ring,size,&rx_buf_pgcnt);if(!size){xdp.data=NULL;
@@ -1200,7 +1194,7 @@ int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget)cleaned_count++;/* skip if it is NOP desc */-if(ice_is_non_eop(rx_ring,rx_desc,skb))+if(ice_is_non_eop(rx_ring,rx_desc))continue;stat_err_bits=BIT(ICE_RX_FLEX_DESC_STATUS0_RXE_S);
@@ -1230,6 +1224,7 @@ int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget)/* send completed skb up the stack */ice_receive_skb(rx_ring,skb,vlan_tag);+skb=NULL;/* update budget accounting */total_rx_pkts++;
@@ -1240,6 +1235,7 @@ int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget)if(xdp_prog)ice_finalize_xdp_rx(rx_ring,xdp_xmit);+rx_ring->skb=skb;ice_update_rx_ring_stats(rx_ring,total_rx_pkts,total_rx_bytes);
@@ -298,6 +297,7 @@ struct ice_ring {structxsk_buff_pool*xsk_pool;/* CL3 - 3rd cacheline starts here */structxdp_rxq_infoxdp_rxq;+structsk_buff*skb;/* CLX - the below items are only accessed infrequently and should be*intheirowncachelineifpossible*/
From: Intel-wired-lan <redacted> On Behalf Of Maciej Fijalkowski
Sent: Monday, January 18, 2021 7:13 AM
To: intel-wired-lan@lists.osuosl.org
Cc: netdev@vger.kernel.org; kuba@kernel.org; bpf@vger.kernel.org; Topel, Bjorn <redacted>; Karlsson, Magnus <magnus.karlsson@intel.com>
Subject: [Intel-wired-lan] [PATCH v3 net-next 06/11] ice: remove redundant checks in ice_change_mtu
dev_validate_mtu checks that mtu value specified by user is not less than min mtu and not greater than max allowed mtu. It is being done before calling the ndo_change_mtu exposed by driver, so remove these redundant checks in ice_change_mtu.
Reviewed-by: Björn Töpel <redacted>
Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
drivers/net/ethernet/intel/ice/ice_main.c | 9 ---------
1 file changed, 9 deletions(-)
Tested-by: Tony Brelinski <redacted> A Contingent Worker at Intel
From: Intel-wired-lan <redacted> On Behalf Of Maciej Fijalkowski
Sent: Monday, January 18, 2021 7:13 AM
To: intel-wired-lan@lists.osuosl.org
Cc: netdev@vger.kernel.org; kuba@kernel.org; bpf@vger.kernel.org; Topel, Bjorn <redacted>; Karlsson, Magnus <magnus.karlsson@intel.com>
Subject: [Intel-wired-lan] [PATCH v3 net-next 05/11] ice: move skb pointer from rx_buf to rx_ring
Similar thing has been done in i40e, as there is no real need for having the sk_buff pointer in each rx_buf. Non-eop frames can be simply handled on that pointer moved upwards to rx_ring.
Reviewed-by: Björn Töpel <redacted>
Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
drivers/net/ethernet/intel/ice/ice_txrx.c | 30 ++++++++++------------- drivers/net/ethernet/intel/ice/ice_txrx.h | 2 +-
2 files changed, 14 insertions(+), 18 deletions(-)
Tested-by: Tony Brelinski <redacted> A Contingent Worker at Intel
From: Intel-wired-lan <redacted> On Behalf Of Maciej Fijalkowski
Sent: Monday, January 18, 2021 7:13 AM
To: intel-wired-lan@lists.osuosl.org
Cc: netdev@vger.kernel.org; kuba@kernel.org; bpf@vger.kernel.org; Topel, Bjorn <redacted>; Karlsson, Magnus <magnus.karlsson@intel.com>
Subject: [Intel-wired-lan] [PATCH v3 net-next 10/11] ice: store the result of ice_rx_offset() onto ice_ring
Output of ice_rx_offset() is based on ethtool's priv flag setting, which when changed, causes PF reset (disables napi, frees irqs, loads different Rx mem model, etc.). This means that within napi its result is constant and there is no reason to call it per each processed frame.
Add new 'rx_offset' field to ice_ring that is meant to hold the
ice_rx_offset() result and use it within ice_clean_rx_irq().
Furthermore, use it within ice_alloc_mapped_page().
Reviewed-by: Björn Töpel <redacted>
Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
drivers/net/ethernet/intel/ice/ice_txrx.c | 43 ++++++++++++----------- drivers/net/ethernet/intel/ice/ice_txrx.h | 1 +
2 files changed, 23 insertions(+), 21 deletions(-)
Tested-by: Tony Brelinski <redacted> A Contingent Worker at Intel