Re: [PATCH] xsk: add cq event

On 2020-11-16 17:12, Xuan Zhuo wrote:

On Mon, 16 Nov 2020 15:31:20 +0100, =?UTF-8?B?QmrDtnJuIFTDtnBlbA==?= [off-list ref] wrote:

quoted

On 2020-11-16 09:10, Xuan Zhuo wrote:

quoted

quoted

When we write all cq items to tx, we have to wait for a new event based

quoted

quoted

on poll to indicate that it is writable. But the current writability is

quoted

quoted

triggered based on whether tx is full or not, and In fact, when tx is

quoted

quoted

dissatisfied, the user of cq's item may not necessarily get it, because it

quoted

quoted

may still be occupied by the network card. In this case, we need to know

quoted

quoted

when cq is available, so this patch adds a socket option, When the user

quoted

quoted

configures this option using setsockopt, when cq is available, a

quoted

quoted

readable event is generated for all xsk bound to this umem.

quoted

quoted

quoted

quoted

I can't find a better description of this event,

quoted

quoted

I think it can also be 'readable', although it is indeed different from

quoted

quoted

the 'readable' of the new data. But the overhead of xsk checking whether

quoted

quoted

cq or rx is readable is small.

quoted

quoted

quoted

quoted

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>

quoted

quoted

Thanks for the patch!

quoted

quoted

I'm not a fan of having two different "readable" event (both Rx and cq).

quoted

Could you explain a bit what the use case is, so I get a better

quoted

understanding.

quoted

quoted

The Tx queues has a back-pressure mechanism, determined of the number of

quoted

elements in cq. Is it related to that?

quoted

quoted

Please explain a bit more what you're trying to solve, and maybe we can

quoted

figure out a better way forward!

quoted

quoted

quoted

Thanks!

quoted

Björn

I want to implement a tool for mass sending. For example, the size of cq is

1024, and I set the size of tx also to 1024, so that I will put all cq in tx at

once, and then I have to wait for an event, come Indicates that there is new

write space or new cq is available.



At present, we can only monitor the event of write able. This indicates whether

tx is full, but in fact, tx is basically not full, because the full state is

very short, and those tx items are being used by the network card. And

epoll_wait will be awakened directly every time, without waiting, but I cannot

get the cq item, so I still cannot successfully send the package again.



Of course, I don't like the "readable" event very much. This is a suitable

one I found in the existing epoll event. ^_^

More questions! By "Mass sending" do you mean maximum throughput, or
does that mean "in very large batches"?

For the latter to do 1k batches, you could increase the Tx/cq buffer
size to say 4k.

For maximum thoughput it's better to use smaller batches (e.g. what the
txpush scenario in samples/xdpsock does).

You're right that even if there's space in the Tx ring, it wont be sent
unless there's sufficient space in the cq ring. Maybe it would make
sense to be more restrictive when triggering the "writable" socket
event? E.g. only trigger it when there's space in Tx *and* sufficient cq
space?


Björn

Thanks.