Re: [PATCH v2 1/1] xfstests: Add Log Attribute Replay test

On 9/1/21 4:36 PM, Dave Chinner wrote:

On Wed, Sep 01, 2021 at 10:10:06PM +0000, Catherine Hoang wrote:

quoted

From: Allison Henderson <redacted>

This patch adds a test to exercise the log attribute error
inject and log replay.  Attributes are added in increaseing
sizes up to 64k, and the error inject is used to replay them
from the log

Signed-off-by: Allison Henderson <redacted>
Signed-off-by: Catherine Hoang <redacted>
---
V2: Updated attr sizes
    Added attr16k test
    Removed rm -f $seqres.full
    Added filtering for SCRATCH_MNT

....

quoted

+_test_attr_replay()
+{
+	attr_name=$1
+	attr_value=$2
+	touch $testfile.1
+
+	echo "Inject error"
+	_scratch_inject_error "larp"
+
+	echo "Set attribute"
+	echo "$attr_value" | ${ATTR_PROG} -s "$attr_name" $testfile.1 2>&1 | \
+			    _filter_scratch
+
+	echo "FS should be shut down, touch will fail"
+	touch $testfile.1 2>&1 | _filter_scratch
+
+	echo "Remount to replay log"
+	_scratch_inject_logprint >> $seqres.full

Huh. That function name has nothing to do with remount or dumping
the log. _scratch_remount_dump_log() would at least describe what it
does (indeed, it is just scratch_unmount; scratch_dump_log;
scratch_mount). Can you follow this up with another patch to
rename _scratch_inject_logprint() to _scratch_remount_dump_log()
and also do the same for the equivalent _test_inject_logprint()
function? They should probably move to common/xfs from
common/inject, too...

Sure, we can do a separate clean up patch for that

quoted

+
+	echo "FS should be online, touch should succeed"
+	touch $testfile.1
+
+	echo "Verify attr recovery"
+	_getfattr --absolute-names $testfile.1 | _filter_scratch
+	$ATTR_PROG -g $attr_name $testfile.1 | md5sum
+
+	echo ""
+}

Ok, so this tests just the "set" operation.

FWIW, there is no need to echo test beahviour descriptions to the
output file. Each of the "echo" statements here should just be
comments.

Ok, we can turn the echos into comments then

quoted

+
+
+# real QA test starts here
+_supported_fs xfs
+
+_require_scratch
+_require_attrs
+_require_xfs_io_error_injection "larp"
+_require_xfs_sysfs debug/larp
+
+# turn on log attributes
+echo 1 > /sys/fs/xfs/debug/larp
+
+_scratch_unmount >/dev/null 2>&1
+
+#attributes of increaseing sizes
+attr16="0123456789ABCDEF"
+attr64="$attr16$attr16$attr16$attr16"
+attr256="$attr64$attr64$attr64$attr64"
+attr1k="$attr256$attr256$attr256$attr256"
+attr4k="$attr1k$attr1k$attr1k$attr1k"
+attr8k="$attr4k$attr4k"
+attr16k="$attr8k$attr8k"
+attr32k="$attr16k$attr16k"
+attr64k="$attr32k$attr32k"
+
+echo "*** mkfs"
+_scratch_mkfs_xfs >/dev/null
+
+echo "*** mount FS"
+_scratch_mount
+
+testfile=$SCRATCH_MNT/testfile
+echo "*** make test file 1"
+
+_test_attr_replay "attr_name1" $attr16
+_test_attr_replay "attr_name2" $attr64
+_test_attr_replay "attr_name3" $attr256
+_test_attr_replay "attr_name4" $attr1k
+_test_attr_replay "attr_name5" $attr4k
+_test_attr_replay "attr_name6" $attr8k
+_test_attr_replay "attr_name7" $attr16k
+_test_attr_replay "attr_name8" $attr32k
+_test_attr_replay "attr_name9" $attr64k

Hmmm - all attributes have different names, so this only tests
the "create new attribute" operation, not the "replace attribute"
or "remove attribute" operations.

Also, why were the given sizes chosen? It seems to me like we should
be selecting the attribute sizes based on the different operations
they trigger.

For an empty 512 byte inode on 4kB block size fs, we have ~300 bytes
available for local attr storage. Hence both attr16 and attr64 will
be stored inline.  attr256 will trigger sf-to-leaf transition with
existing entries.  attr1k will do a leaf internal addition. attr4k
will be stored externally as a remote attr, as will all the
remaining larger attrs.

Hence this doesn't test the following cases:
- empty to leaf transition on first attr insert
- remote xattr insertion when empty
- leaf split/addition due to filling a leaf block
- extent format to btree format transistion (i.e. tree level
   increase)

IOWs, for a 512 byte inode and 4kB block size fs, the tests really
need to be:

- empty, add inline attr	(64 bytes)
- empty, add internal attr	(1kB)
- empty, add remote attr	(64kB)
- inline, add inline attr	(64 bytes)
- inline, add internal attr	(1kB)
- inline, add remote attr	(64kB)
- extent, add internal attr	(1kB)
- extent, add multiple internal attr (inject error on split operation)
- extent, add multiple internal attr (inject error on fork
				  transition to btree format operation)
- extent, add remote attr	(64kB)
- btree, add multiple internal	(1kB)
- btree, add remote attr	(64kB)

This covers all the different attr fork storage forms and
transitions between the different forms.

Ok, so if I'm understanding this description correctly, I think we can 
just add a file name parameter to the _test_attr_replay function, and 
then modify the test calls to look something like this:

# test empty attr add
touch empty_file1
touch empty_file2
touch empty_file3
_test_attr_replay empty_file1 "attr_name" $attr64
_test_attr_replay empty_file2 "attr_name" $attr1k
_test_attr_replay empty_file3 "attr_name" $attr64k

# test inline attr add
touch inline_file
_test_attr_replay inline_file "attr_name1" $attr64
_test_attr_replay inline_file "attr_name2" $attr1k
_test_attr_replay inline_file "attr_name3" $attr64k
_test_attr_replay inline_file "attr_name4" $attr1k


# test split on leaf
touch leaf_file
echo "$attr16$att64" | ${ATTR_PROG} -s "attr_name" leaf_file 2>&1 | \
			    _filter_scratch
_test_attr_replay inline_file "attr_name2" $attr256


# test fork transition
touch fork_file
echo "$attr1k" | ${ATTR_PROG} -s "attr_name" fork_file 2>&1 | \
			    _filter_scratch
_test_attr_replay fork_file "attr_name2" $4k


#test remote attr
touch remote_file
_test_attr_replay remote_file "attr_name1" $attr64k
_test_attr_replay remote_file "attr_name2" $attr1k
_test_attr_replay remote_file "attr_name3" $attr64k

Does that reflect what you are meaning to describe?

We then need to cover the same cases but in reverse for attr removal
(e.g. recovery of leaf merge operations, btree to extent form
conversion, etc).

Sure, we can add a _test_attr_rmv_replay.  Maybe _test_attr_replay 
should be renamed _test_attr_set_replay too.

We also need to have coverage of attr overwrite recovery of all the
attr formats (shortform, leaf internal and remote) because these
both add and remove attributes of the same name. We probably want
different points of error injection for these so that we can force
it to recover from different points in the replace operation...

So you want more error tags?  Maybe one for the shortform, leaf and node?

Thanks!
Allison

Cheers,

Dave.