On Thu, Feb 03, 2022 at 12:10:27PM -0800, Jakub Kicinski wrote:

On Thu, 3 Feb 2022 20:21:28 +0200 Vladimir Oltean wrote:

quoted

To my knowledge, when you call dev_queue_xmit(), the skb is no longer
yours, end of story, it doesn't matter whether you increase the refcount
on it or not. The DSA master may choose to do whatever it wishes with
that buffer after its TX completion interrupt fires: it may not call
napi_consume_skb() but directly recycle that buffer in its pool of RX
buffers, as part of some weird buffer recycling scheme. So you'll think
that the buffer is yours, but it isn't, because the driver hasn't
returned it to the allocator, and your writes for the next packet may be
concurrent with some RX DMA transactions. I don't have a mainline
example to give you, but I've seen the pattern, and I don't think it's
illegal (although of course, I stand to be corrected if necessary).

Are we talking about holding onto the Tx skb here or also recycling 
the Rx one? Sorry for another out of context comment in advance..

Here we are talking about tx skb. (We can't really operate on the
received skb handled by the tagger)
What I want to improve is the fact that we alloc 3 very small skb that
will have only some part of them modified and the rest equal on all 3
skb. So my idea is find a way to preallocate a space and build a skb
around it. In short as we force a 1:1 comunication where we send an skb
and we wait for the response to be processed, we can reuse the same skb
everytime and just modify the data in it. I'm asking if anyone have some
hint/direction without proposing a patch that is a big massive hack that
will be NACK in 0.1 ms LOL

AFAIK in theory shared skbs are supposed to be untouched or unshared
explicitly by the driver on Tx. pktgen takes advantage of it.
We have IFF_TX_SKB_SHARING. 

Will check how this flags is used. If you have any hint about this feel
free to suggest it.

In practice everyone gets opted into SKB_SHARING because ether_setup()
sets the flag. A lot of drivers are not aware of the requirement and
will assume full ownership (and for example use skb->cb[]) :/

Consider that currently this will be used by stmmac driver, brcm driver
and someother atheros driver.

I don't think there is any Tx completion -> Rx pool recycling scheme
inside the drivers (if that's what you described).

-- 
	Ansuel