On 12/30, Martin KaFai Lau wrote:

On Mon, Dec 21, 2020 at 02:22:41PM -0800, Song Liu wrote:

quoted

On Thu, Dec 17, 2020 at 9:24 AM Stanislav Fomichev [off-list ref]  

wrote:

quoted

quoted

When we attach a bpf program to cgroup/getsockopt any other  

getsockopt()

quoted

quoted

syscall starts incurring kzalloc/kfree cost. While, in general, it's
not an issue, sometimes it is, like in the case of  

TCP_ZEROCOPY_RECEIVE.

quoted

quoted

TCP_ZEROCOPY_RECEIVE (ab)uses getsockopt system call to implement
fastpath for incoming TCP, we don't want to have extra allocations in
there.

Let add a small buffer on the stack and use it for small (majority)
{s,g}etsockopt values. I've started with 128 bytes to cover
the options we care about (TCP_ZEROCOPY_RECEIVE which is 32 bytes
currently, with some planned extension to 64 + some headroom
for the future).

I don't really know the rule of thumb, but 128 bytes on stack feels too  

big to

quoted

me. I would like to hear others' opinions on this. Can we solve the  

problem

quoted

with some other mechanisms, e.g. a mempool?

It seems the do_tcp_getsockopt() is also having "struct  
tcp_zerocopy_receive"
in the stack.  I think the buf here is also mimicking
"struct tcp_zerocopy_receive", so should not cause any
new problem.

Good point!

However, "struct tcp_zerocopy_receive" is only 40 bytes now.  I think it
is better to have a smaller buf for now and increase it later when the
the future needs in "struct tcp_zerocopy_receive" is also upstreamed.

I can lower it to 64. Or even 40?

I can also try to add something like BUILD_BUG_ON(sizeof(struct
tcp_zerocopy_receive) < BPF_SOCKOPT_KERN_BUF_SIZE) to make sure this
buffer gets adjusted whenever we touch tcp_zerocopy_receive.