On Fri, Aug 14, 2020 at 08:27:13AM +0000, Damien Le Moal wrote:

quoted

O_APPEND pretty much implies out of order, as there is no way for an
application to know which thread wins the race to write the next chunk.

Yes and no. If the application threads do not synchronize their calls to
io_submit(), then yes indeed, things can get out of order. But if the
application threads are synchronized, then the offset set for each append AIO
will be in sequence of submission, so the user will not see its writes
completing at different write offsets than this implied offsets.

Nothing gurantees any kind of ordering for two separate io_submit calls.
The kernel may delay one of them for any reason.

Now if you are doing two fully synchronous write calls on an
O_APPEND fd, yes they are serialized.  But using Zone Append won't
change that.