Re: LTP test df01.sh detected different size of loop device in v5.19

Hi Dave,

On Fri, Aug 12, 2022 at 03:20:37PM +0200, Petr Vorel wrote:

quoted

Hi all,

quoted

LTP test df01.sh found different size of loop device in v5.19.
Test uses loop device formatted on various file systems, only XFS fails.
It randomly fails during verifying that loop size usage changes:

quoted

grep ${TST_DEVICE} output | grep -q "${total}.*${used}" [1]

quoted

How to reproduce:
# PATH="/opt/ltp/testcases/bin:$PATH" df01.sh -f xfs # it needs several tries to hit

quoted

df saved output:
Filesystem     1024-blocks    Used Available Capacity Mounted on
...
/dev/loop0          256672   16208    240464       7% /tmp/LTP_df01.1kRwoUCCR7/mntpoint
df output:
Filesystem     1024-blocks    Used Available Capacity Mounted on
...
tmpfs               201780       0    201780       0% /run/user/0
/dev/loop0          256672   15160    241512       6% /tmp/LTP_df01.1kRwoUCCR7/mntpoint
=> different size
df01 4 TFAIL: 'df -k -P' failed, not expected.

Yup, most likely because we changed something in XFS related to
internal block reservation spaces. That is, the test is making
fundamentally flawed assumptions about filesystem used space
accounting.

It is wrong to assuming that the available capacity of a given empty
filesystem will never change.  Assumptions like this have been
invalid for decades because the available space can change based on
the underlying configuration or the filesystem. e.g. different
versions of mkfs.xfs set different default parameters and so simply
changing the version of xfsprogs you use between the two comparision
tests will make it fail....

And, well, XFS also has XFS_IOC_{GS}ET_RESBLKS ioctls that allow
userspace to change the amount of reserved blocks. They were
introduced in 1997, and since then we've changed the default
reservation the filesystem takes at least a dozen times.

Thanks a lot for valuable info.

quoted

quoted

It might be a false positive / bug in the test, but it's at least a changed behavior.

Yup, any test that assumes "available space" does not change from
kernel version to kernel version is flawed. There is no guarantee
that this ever stays the same, nor that it needs to stay the same.

I'm sorry I was not clear. Test [1] does not measure "available space" between
kernel releases. It just run df command with parameters, saves it's output
and compares "1024-blocks" and "Used" columns of df output with stat output:

		local total=$(stat -f mntpoint --printf=%b)
		local free=$(stat -f mntpoint --printf=%f)
		local used=$((total-free))

		local bsize=$(stat -f mntpoint --printf=%s)
		total=$((($total * $bsize + 512)/ 1024))
		used=$((($used * $bsize + 512) / 1024))

And comparison with "$used" is what sometimes fails.

BTW this happens on both distros when loop device is on tmpfs. I'm trying to
trigger it on ext4 and btrfs, not successful so far. Looks like it's tmpfs
related.

If that's really expected, we might remove this check for used for XFS
(not sure if check only for total makes sense).

quoted

quoted

I was able to reproduce it on v5.19 distro kernels (openSUSE, Debian).
I haven't bisected (yet), nor checked Jens' git tree (maybe it has been fixed).

quoted

Forget to note dmesg "operation not supported error" warning on *each* run (even
successful) on affected v5.19:
[ 5097.594021] loop0: detected capacity change from 0 to 524288
[ 5097.658201] operation not supported error, dev loop0, sector 262192 op 0x9:(WRITE_ZEROES) flags 0x8000800 phys_seg 0 prio class 0
[ 5097.675670] XFS (loop0): Mounting V5 Filesystem
[ 5097.681668] XFS (loop0): Ending clean mount
[ 5097.956445] XFS (loop0): Unmounting Filesystem

That warning is from mkfs attempting to use fallocate(ZERO_RANGE) to
offload the zeroing of the journal to the block device. It would
seem that the loop device image file is being hosted on a filesystem
that does not support the fallocate() ZERO_RANGE (or maybe
PUNCH_HOLE) operation. That warning should simply be removed - it
serves no useful purpose to a user...

Interesting. Which one of these two is not supported on tmpfs?

Kind regards,
Petr

CHeers,

Dave.

[1] https://github.com/linux-test-project/ltp/blob/f42f6f3b4671f447b743afe8612917ba4362b8a6/testcases/commands/df/df01.sh