On Mon, Oct 25, 2021 at 11:57 PM Jan Kara [off-list ref] wrote:

On Sat 23-10-21 10:06:24, Zhengyuan Liu wrote:

quoted

On Fri, Oct 22, 2021 at 5:31 PM Jan Kara [off-list ref] wrote:

quoted

quoted

quoted

quoted

quoted

Can you post output of "dumpe2fs -h <device>" for the filesystem where the
problem happens? Thanks!

Sure, the output is:

# dumpe2fs -h /dev/sda3
dumpe2fs 1.45.3 (14-Jul-2019)
Filesystem volume name:   <none>
Last mounted on:          /data
Filesystem UUID:          09a51146-b325-48bb-be63-c9df539a90a1
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode dir_index
filetype needs_recovery sparse_super large_file

Thanks for the data. OK, a filesystem without extents. Does your test by
any chance try to do direct IO to a hole in a file? Because that is not
(and never was) supported without extents. Also the fact that you don't see
the problem with ext4 (which means extents support) would be pointing in
that direction.

I am not sure if it trys to do direct IO to a hole or not, is there any
way to check?  If you have a simple test to reproduce please let me know,
we are glad to try.

Can you enable following tracing?

Sure, but let's confirm before doing that, it seems Ext4 doesn't
support iomap in
V4.19 which could also reproduce the problem, so if it is necessary to
do the following
tracing? or should we modify the tracing if under V4.19?

Well, iomap is just a different generic framework for doing direct IO. The
fact that you can observe the problem both with iomap and the old direct IO
framework is one of the reasons why I think the problem is actually that
the file has holes (unallocated space in the middle).

quoted

quoted

echo 1 >/sys/kernel/debug/tracing/events/ext4/ext4_ind_map_blocks_exit/enable
echo iomap_dio_rw >/sys/kernel/debug/tracing/set_ftrace_filter
echo "function_graph" >/sys/kernel/debug/tracing/current_tracer

If you want to trace a kernel were ext4 direct IO path is not yet
converted to iomap framework you need to replace tracing of iomap_dio_rw
with:

echo __blockdev_direct_IO >/sys/kernel/debug/tracing/set_ftrace_filter

Sorry for the late reply, because it is hard to reproduce when enabling ftrace.
I'll post here right after getting useful data.

quoted

quoted

And then gather output from /sys/kernel/debug/tracing/trace_pipe. Once the
problem reproduces, you can gather the problematic file name from dmesg, find
inode number from "stat <filename>" and provide that all to me? Thanks!