Hi,

We would like to revive the discussion on zerocopy sendfile support for
TLS offload:

https://lore.kernel.org/all/1594551195-3579-1-git-send-email-borisp@mellanox.com/ (local)

I believe we can resolve the concern about correctness vs performance
expressed in the previous discussion.

Historically, sendfile implementations for TCP and TLS allow changing
the underlying file while sending it. The connection is not disrupted,
but besides that there aren't many guarantees: the contents of the
received file may be a mix of the old and new versions (a lost middle
packet may be retransmitted with the new data, after the packets
following it were received with the old data). The goal is to preserve
this behavior for all existing users.

Zerocopy TLS sendfile provides even fewer guarantees: if a part of a TLS
record is being retransmitted on TCP level, while the file is being
changed, the receiver may get a TLS record with bad signature and close
the connection. That means we can't simply replace the current behavior
with zerocopy.

On the other hand, even with such a limitation, zerocopy TLS sendfile is
extremely useful in a very common use case of serving static files over
HTTPS. Web files normally have formats that become damaged and useless
after arbitrary partial updates. From that perspective, receiving a
damaged file or closing the connection is equally bad for the client.
Admins should normally avoid updating static files without stopping the
server, but even if they don't follow this recommendation, zerocopy
itself doesn't impair user experience compared to regular sendfile. At
the same time, it boosts the TX speed by up to 25% and reduces CPU load
by up to 12.5%.

Given that we would like to keep the current sendfile behavior for all
existing users, while still being able to use acceleration of zerocopy
in applicable scenarios, I suggest considering including zerocopy TLS
sendfile as an opt-in feature. The default will always be non-zerocopy,
and there will be no global sysctl knob to change it, so that it won't
be possible to break existing applications. The users willing to use
zerocopy deliberately will set a flag, for example, by setsockopt (or
any other mechanism).

Most importantly, there is no concern of violating kernel integrity. The
userspace won't be able to crash the kernel or bypass protection using
the new feature. It also won't be able to trick the kernel into
transmitting some data that the userspace can't send using a regular TCP
socket.

What do you think about these points? Does zerocopy TLS sendfile have a
future in the kernel as an opt-in feature?

Thanks,
Max