On 18 Sep 2021 at 05:33, Darrick J. Wong wrote:

On Thu, Sep 16, 2021 at 03:36:35PM +0530, Chandan Babu R wrote:

quoted

The commit xfs: fix inode fork extent count overflow
(3f8a4f1d876d3e3e49e50b0396eaffcc4ba71b08) mentions that 10 billion
data fork extents should be possible to create. However the
corresponding on-disk field has a signed 32-bit type. Hence this
patchset extends the per-inode data extent counter to 64 bits out of
which 48 bits are used to store the extent count. 

Also, XFS has an attr fork extent counter which is 16 bits wide. A
workload which,
1. Creates 1 million 255-byte sized xattrs,
2. Deletes 50% of these xattrs in an alternating manner,
3. Tries to insert 400,000 new 255-byte sized xattrs
   causes the xattr extent counter to overflow.

Dave tells me that there are instances where a single file has more
than 100 million hardlinks. With parent pointers being stored in
xattrs, we will overflow the signed 16-bits wide xattr extent counter
when large number of hardlinks are created. Hence this patchset
extends the on-disk field to 32-bits.

The following changes are made to accomplish this,
1. A new incompat superblock flag to prevent older kernels from mounting
   the filesystem. This flag has to be set during mkfs time.
2. A new 64-bit inode field is created to hold the data extent
   counter.
3. The existing 32-bit inode data extent counter will be used to hold
   the attr fork extent counter.

The patchset has been tested by executing xfstests with the following
mkfs.xfs options,
1. -m crc=0 -b size=1k
2. -m crc=0 -b size=4k
3. -m crc=0 -b size=512
4. -m rmapbt=1,reflink=1 -b size=1k
5. -m rmapbt=1,reflink=1 -b size=4k

Each of the above test scenarios were executed on the following
combinations (For V4 FS test scenario, the last combination
i.e. "Patched (enable extcnt64bit)", was omitted).
|-------------------------------+-----------|
| Xfsprogs                      | Kernel    |
|-------------------------------+-----------|
| Unpatched                     | Patched   |
| Patched (disable extcnt64bit) | Unpatched |
| Patched (disable extcnt64bit) | Patched   |
| Patched (enable extcnt64bit)  | Patched   |
|-------------------------------+-----------|

I have also written a test (yet to be converted into xfstests format)
to check if the correct extent counter fields are updated with/without
the new incompat flag. I have also fixed some of the existing fstests
to work with the new extent counter fields.

Increasing data extent counter width also causes the maximum height of
BMBT to increase. This requires that the macro XFS_BTREE_MAXLEVELS be
updated with a larger value. However such a change causes the value of
mp->m_rmap_maxlevels to increase which in turn causes log reservation
sizes to increase and hence a modified XFS driver will fail to mount
filesystems created by older versions of mkfs.xfs.

Hence this patchset is built on top of Darrick's btree-dynamic-depth
branch which removes the macro XFS_BTREE_MAXLEVELS and computes
mp->m_rmap_maxlevels based on the size of an AG.

I forward-ported /just/ that branch to a 5.16 dev branch and will send
that out, in case you wanted to add it to the head of your dev branch
and thereby escape relying on the bajillion patches in djwong-dev.

Thanks for doing that. I will rebase my patchset on top of "xfs: support
dynamic btree cursor height" series.

-- 
chandan