On Jun 20, 2017, at 12:20 PM, Darrick J. Wong [off-list ref] wrote:

quoted

Now that you have an idea of what caused the issue, is there some way
to (easily or not) reproduce it, ideally in userspace?  Part of the
problem we're having is that we can't reproduce it on a test machine
so are forced to wait for it to happen in the field under load, where
it inconveniences users.

I know what caused the crash, but I don't know what caused the hole in
the xattr fork that caused the crash.  It would appear that the xattr
hash tree contains a pointer to an xattr fork offset that is no longer
mapped, but I don't know how we end up with a dangling pointer --
whether this is corruption going on in the attribute fork, or merely a
bug in the routine that tears down the attribute fork and accidentally
tries to delete something twice.

Unfortunately, the metadumps you've both sent me don't seem to contain
any inodes with sparse attribute forks, so evidently something cleaned
up the filesystem before the metadump was taken.

The filesystems were automatically mounted after the box rebooted.  Could the mount (via replaying the journal) have cleaned things up?

I have a reproduction case for this now - it takes several hours, but can reliably cause this panic.  I will send you a new metadump that was not mounted after the failure.

It might help to know a little more about what's going on with the
xattrs that are being attached to those files.  A fair number of the
inodes seem to have extended attributes with 2k values attached.  Does
your workload involve adding and deleting a large number of xattrs?  A
number of large xattrs?  A large number of large xattrs?

All of the above.  This is Samba with the alternate data streams module (vfs_streams_xattr), which stores the ADS in xattrs.  It makes for some large xattrs.

David