From: Eric Biggers <ebiggers@kernel.org> Date: 2021-01-05 00:56:05
Hello,
This patchset fixes the lazytime bug which I reported last year
(https://lore.kernel.org/r/20200306004555.GB225345@gmail.com). This bug
causes inodes with dirty timestamps to remain dirty after a sync, which
causes the inodes to be unnecessarily written again, and also causes the
FS_IOC_REMOVE_ENCRYPTION_KEY ioctl to not work properly. This bug is
causing xfstest generic/580 to fail when lazytime is enabled.
Ted and Christoph proposed fixes for this. Ted's fix
(https://lore.kernel.org/r/20200307020043.60118-1-tytso@mit.edu) changed
the call to mark_inode_dirty_sync(inode) in __writeback_single_inode()
to ->dirty_inode(inode, I_DIRTY_TIME_EXPIRED). However this would have
broken XFS, which wants an I_DIRTY_SYNC notification. Also, people
preferred a larger rework involving adding a ->lazytime_expired method.
Christoph's fix
(https://lore.kernel.org/r/20200325122825.1086872-3-hch@lst.de)
introduced ->lazytime_expired, but it wasn't correct because it didn't
consider cases in which timestamps are force-expired.
To resolve this, I propose that we first fix the bug by making
__writeback_single_inode() do an I_DIRTY_SYNC notification if the
timestamps expired (patch #1).
Then, the remaining patches introduce ->lazytime_expired and make XFS
use it. They also clean up various things, such as improving comments.
Also, it turns out that lazytime on XFS is broken because it doesn't
actually write timestamps to disk after a sync() or after 24 hours.
This is fixed by the patch to switch XFS to use ->lazytime_expired.
I've written an xfstest which reproduces this bug.
This patchset applies to v5.11-rc2.
Eric Biggers (13):
fs: avoid double-writing inodes on lazytime expiration
gfs2: don't worry about I_DIRTY_TIME in gfs2_fsync()
fs: only specify I_DIRTY_TIME when needed in generic_update_time()
fat: only specify I_DIRTY_TIME when needed in fat_update_time()
fs: don't call ->dirty_inode for lazytime timestamp updates
fs: pass only I_DIRTY_INODE flags to ->dirty_inode
fs: correctly document the inode dirty flags
ext4: simplify i_state checks in __ext4_update_other_inode_time()
fs: drop redundant checks from __writeback_single_inode()
fs: clean up __mark_inode_dirty() a bit
fs: add a lazytime_expired method
xfs: remove a stale comment from xfs_file_aio_write_checks()
xfs: implement lazytime_expired
Documentation/filesystems/locking.rst | 2 +
Documentation/filesystems/vfs.rst | 15 ++++-
fs/ext4/inode.c | 20 ++----
fs/f2fs/super.c | 3 -
fs/fat/misc.c | 21 +++---
fs/fs-writeback.c | 94 +++++++++++++++++++--------
fs/gfs2/file.c | 4 +-
fs/gfs2/super.c | 2 -
fs/inode.c | 40 ++++++------
fs/sync.c | 2 +-
fs/xfs/xfs_file.c | 6 --
fs/xfs/xfs_super.c | 12 +---
include/linux/fs.h | 25 +++++--
13 files changed, 143 insertions(+), 103 deletions(-)
base-commit: e71ba9452f0b5b2e8dc8aa5445198cd9214a6a62
--
2.30.0
From: Eric Biggers <ebiggers@kernel.org> Date: 2021-01-05 00:56:03
From: Eric Biggers <redacted>
When lazytime is enabled and an inode with dirty timestamps is being
expired, either due to dirtytime_expire_interval being exceeded or due
to a sync or syncfs system call, we need to inform the filesystem that
the inode is dirty so that the inode's timestamps can be copied out to
the on-disk data structures. That's because if the filesystem supports
lazytime, it will have ignored the ->dirty_inode(inode, I_DIRTY_TIME)
notification when the timestamp was modified in memory.
Currently this is accomplished by calling mark_inode_dirty_sync() from
__writeback_single_inode(). However, this has the unfortunate side
effect of also putting the inode the writeback list. That's not
appropriate in this case, since the inode is already being written.
That causes the inode to remain dirty after a sync. Normally that's
just wasteful, as it causes the inode to be written twice. But when
fscrypt is used this bug also partially breaks the
FS_IOC_REMOVE_ENCRYPTION_KEY ioctl, as the ioctl reports that files are
still in-use when they aren't. For more details, see the original
report at https://lore.kernel.org/r/20200306004555.GB225345@gmail.com
Fix this by calling ->dirty_inode(inode, I_DIRTY_SYNC) directly instead
of mark_inode_dirty_sync().
This fixes xfstest generic/580 when lazytime is enabled.
A later patch will introduce a ->lazytime_expired method to cleanly
separate out the lazytime expiration case, in particular for XFS which
uses the VFS-level dirtiness tracking only for lazytime. But that's
separate from fixing this bug. Also, note that XFS will incorrectly
ignore the I_DIRTY_SYNC notification from __writeback_single_inode()
both before and after this patch, as I_DIRTY_TIME was already cleared in
i_state. Later patches will fix this separate bug.
Fixes: 0ae45f63d4ef ("vfs: add support for a lazytime mount option")
Cc: stable@vger.kernel.org
Signed-off-by: Eric Biggers <redacted>
---
fs/fs-writeback.c | 17 ++++++++++++++---
1 file changed, 14 insertions(+), 3 deletions(-)
@@ -1509,11 +1509,22 @@ __writeback_single_inode(struct inode *inode, struct writeback_control *wbc)spin_unlock(&inode->i_lock);-if(dirty&I_DIRTY_TIME)-mark_inode_dirty_sync(inode);/* Don't write the inode if only I_DIRTY_PAGES was set */if(dirty&~I_DIRTY_PAGES){-interr=write_inode(inode,wbc);+interr;++/*+*Iftheinodeisbeingwrittenduetoalazytimetimestamp+*expiration,thenthefilesystemneedstobenotifiedaboutit+*sothate.g.thefilesystemcanupdateon-diskfieldsand+*journalthetimestampupdate.Justcallingwrite_inode()+*isn'tenough.Don'tcallmark_inode_dirty_sync(),asthat+*wouldputtheinodebackonthedirtylist.+*/+if((dirty&I_DIRTY_TIME)&&inode->i_sb->s_op->dirty_inode)+inode->i_sb->s_op->dirty_inode(inode,I_DIRTY_SYNC);++err=write_inode(inode,wbc);if(ret==0)ret=err;}
From: Eric Biggers <ebiggers@kernel.org> Date: 2021-01-05 00:56:04
From: Eric Biggers <redacted>
As was done for generic_update_time(), only pass I_DIRTY_TIME to
__mark_inode_dirty() when the inode's timestamps were actually updated
and lazytime is enabled. This avoids a weird edge case where
I_DIRTY_TIME could be set in i_state when lazytime isn't enabled.
Signed-off-by: Eric Biggers <redacted>
---
fs/fat/misc.c | 21 +++++++++++----------
1 file changed, 11 insertions(+), 10 deletions(-)
From: Eric Biggers <ebiggers@kernel.org> Date: 2021-01-05 00:56:04
From: Eric Biggers <redacted>
generic_update_time() always passes I_DIRTY_TIME to
__mark_inode_dirty(), which doesn't really make sense because (a)
generic_update_time() might be asked to do only an i_version update, not
also a timestamps update; and (b) I_DIRTY_TIME is only supposed to be
set in i_state if the filesystem has lazytime enabled, so using it
unconditionally in generic_update_time() is inconsistent.
As a result there is a weird edge case where if only an i_version update
was requested (not also a timestamps update) but it is no longer needed
(i.e. inode_maybe_inc_iversion() returns false), then I_DIRTY_TIME will
be set in i_state even if the filesystem isn't mounted with lazytime.
Fix this by only passing I_DIRTY_TIME to __mark_inode_dirty() if the
timestamps were updated and the filesystem has lazytime enabled.
Signed-off-by: Eric Biggers <redacted>
---
fs/inode.c | 38 ++++++++++++++++++++------------------
1 file changed, 20 insertions(+), 18 deletions(-)
From: Eric Biggers <ebiggers@kernel.org> Date: 2021-01-05 00:56:05
From: Eric Biggers <redacted>
The I_DIRTY_TIME flag is primary used within the VFS, and there's no
reason for ->fsync() implementations to do anything with it. This is
because when !datasync, the VFS will expire dirty timestamps before
calling ->fsync(). (See vfs_fsync_range().) This turns I_DIRTY_TIME
into I_DIRTY_SYNC.
Therefore, change gfs2_fsync() to not check for I_DIRTY_TIME.
Signed-off-by: Eric Biggers <redacted>
---
fs/gfs2/file.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
From: Eric Biggers <ebiggers@kernel.org> Date: 2021-01-05 00:56:45
From: Eric Biggers <redacted>
->dirty_inode is now only called when I_DIRTY_INODE (I_DIRTY_SYNC and/or
I_DIRTY_DATASYNC) is set. However it may still be passed other dirty
flags at the same time, provided that these other flags happened to be
passed to __mark_inode_dirty() at the same time as I_DIRTY_INODE.
This doesn't make sense because there is no reason for filesystems to
care about these extra flags. Nor are filesystems notified about all
updates to these other flags.
Therefore, mask the flags before passing them to ->dirty_inode.
Also properly document ->dirty_inode in vfs.rst.
Signed-off-by: Eric Biggers <redacted>
---
Documentation/filesystems/vfs.rst | 5 ++++-
fs/fs-writeback.c | 2 +-
2 files changed, 5 insertions(+), 2 deletions(-)
@@ -270,7 +270,10 @@ or bottom half). ->alloc_inode.``dirty_inode``- this method is called by the VFS to mark an inode dirty.+ this method is called by the VFS when an inode is marked dirty.+ This is specifically for the inode itself being marked dirty,+ not its data. If the update needs to be persisted by fdatasync(),+ then I_DIRTY_DATASYNC will be set in the flags argument.``write_inode`` this method is called when the VFS needs to write an inode to
From: Eric Biggers <ebiggers@kernel.org> Date: 2021-01-05 00:56:45
From: Eric Biggers <redacted>
Implement the new ->lazytime_expired method to get notified of lazytime
timestamp expirations, instead of relying on ->dirty_inode(inode,
I_DIRTY_SYNC) which is potentially ambiguous.
This fixes a bug where XFS didn't write lazytime timestamps to disk upon
a sync(), or after 24 hours (dirtytime_expire_interval * 2). This is
because it only wrote the timestamps if I_DIRTY_TIME was set in i_state.
But actually when an inode's timestamps expire without the inode being
marked I_DIRTY_SYNC first, then ->dirty_inode isn't called until
__writeback_single_inode() has already cleared I_DIRTY_TIME in i_state.
The new ->lazytime_expired method is unambiguous, so it removes any need
to check for I_DIRTY_TIME, which avoids this bug.
I've written an xfstest which reproduces this bug.
Fixes: c3b1b13190ae ("xfs: implement the lazytime mount option")
Signed-off-by: Eric Biggers <redacted>
---
fs/xfs/xfs_super.c | 12 +++---------
1 file changed, 3 insertions(+), 9 deletions(-)
From: Eric Biggers <ebiggers@kernel.org> Date: 2021-01-05 00:56:45
From: Eric Biggers <redacted>
The documentation for I_DIRTY_SYNC and I_DIRTY_DATASYNC is a bit
misleading, and I_DIRTY_TIME isn't documented at all. Fix this.
Signed-off-by: Eric Biggers <redacted>
---
include/linux/fs.h | 18 +++++++++++++-----
1 file changed, 13 insertions(+), 5 deletions(-)
From: Eric Biggers <ebiggers@kernel.org> Date: 2021-01-05 00:56:45
From: Eric Biggers <redacted>
I_DIRTY_TIME and I_DIRTY_INODE are mutually exclusive in i_state. So
after seeing that I_DIRTY_TIME is set, there's no point in checking for
I_DIRTY_INODE, as it must be clear.
Separately, wbc->for_sync implies wbc->sync_mode == WB_SYNC_ALL.
So there's no need to check for both. Just check for WB_SYNC_ALL.
Signed-off-by: Eric Biggers <redacted>
---
fs/fs-writeback.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
From: Eric Biggers <ebiggers@kernel.org> Date: 2021-01-05 00:56:46
From: Eric Biggers <redacted>
Add a lazytime_expired method to 'struct super_operations'. Filesystems
can implement this to be notified when an inode's lazytime timestamps
have expired and need to be written to disk.
This avoids any potential ambiguity with
->dirty_inode(inode, I_DIRTY_SYNC), which can also mean a generic
dirtying of the inode, not just a lazytime timestamp expiration.
In particular, this will be useful for XFS.
If not implemented, then ->dirty_inode(inode, I_DIRTY_SYNC) continues to
be called.
Note that there are three cases where we have to make sure to call
lazytime_expired():
- __writeback_single_inode(): inode is being written now
- vfs_fsync_range(): inode is going to be synced
- iput(): inode is going to be evicted
In the latter two cases, the inode still needs to be put on the
writeback list. So, we can't just replace the calls to
mark_inode_dirty_sync() with lazytime_expired(). Instead, add a new
flag I_DIRTY_TIME_EXPIRED which can be passed to __mark_inode_dirty().
It's like I_DIRTY_SYNC, except it causes the filesystem to be notified
of a lazytime expiration rather than a generic I_DIRTY_SYNC.
Signed-off-by: Eric Biggers <redacted>
---
Documentation/filesystems/locking.rst | 2 ++
Documentation/filesystems/vfs.rst | 10 ++++++++++
fs/fs-writeback.c | 27 ++++++++++++++++++++++-----
fs/inode.c | 2 +-
fs/sync.c | 2 +-
include/linux/fs.h | 7 ++++++-
6 files changed, 42 insertions(+), 8 deletions(-)
@@ -231,6 +231,7 @@ filesystem. As of kernel 2.6.22, the following members are defined: void (*destroy_inode)(struct inode *); void (*dirty_inode) (struct inode *, int flags);+ void (*lazytime_expired) (struct inode *); int (*write_inode) (struct inode *, int); void (*drop_inode) (struct inode *); void (*delete_inode) (struct inode *);
@@ -275,6 +276,15 @@ or bottom half). not its data. If the update needs to be persisted by fdatasync(), then I_DIRTY_DATASYNC will be set in the flags argument.+``lazytime_expired``+ when the lazytime mount option is enabled, this method is+ called when an inode's in-memory updated timestamps have+ expired and thus need to be written to disk. This happens+ when the timestamps have been in memory for too long, when the+ inode is going to be evicted, or when userspace triggers a+ sync. If this method is not implemented, then+ ->dirty_inode(inode, I_DIRTY_SYNC) is called instead.+``write_inode`` this method is called when the VFS needs to write an inode to disc. The second parameter indicates whether the write should
From: Eric Biggers <ebiggers@kernel.org> Date: 2021-01-05 00:56:46
From: Eric Biggers <redacted>
Since I_DIRTY_TIME and I_DIRTY_INODE are mutually exclusive in i_state,
there's no need to check for I_DIRTY_TIME && !I_DIRTY_INODE. Just check
for I_DIRTY_TIME.
Signed-off-by: Eric Biggers <redacted>
---
fs/ext4/inode.c | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)
From: Eric Biggers <ebiggers@kernel.org> Date: 2021-01-05 00:56:46
From: Eric Biggers <redacted>
The comment in xfs_file_aio_write_checks() about calling file_modified()
after dropping the ilock doesn't make sense, because the code that
unconditionally acquires and drops the ilock was removed by
commit 467f78992a07 ("xfs: reduce ilock hold times in
xfs_file_aio_write_checks").
Remove this outdated comment.
Signed-off-by: Eric Biggers <redacted>
---
fs/xfs/xfs_file.c | 6 ------
1 file changed, 6 deletions(-)
From: Eric Biggers <ebiggers@kernel.org> Date: 2021-01-05 00:56:47
From: Eric Biggers <redacted>
Improve some comments, and don't bother checking for the I_DIRTY_TIME
flag in the case where we just cleared it.
Also, warn if I_DIRTY_TIME and I_DIRTY_PAGES are passed to
__mark_inode_dirty() at the same time, as this case isn't handled.
Signed-off-by: Eric Biggers <redacted>
---
fs/fs-writeback.c | 49 +++++++++++++++++++++++++++++------------------
1 file changed, 30 insertions(+), 19 deletions(-)
From: Eric Biggers <ebiggers@kernel.org> Date: 2021-01-05 00:56:47
From: Eric Biggers <redacted>
There is no need to call ->dirty_inode for lazytime timestamp updates
(i.e. for __mark_inode_dirty(I_DIRTY_TIME)), since by the definition of
lazytime, filesystems must ignore these updates. Filesystems only need
to care about the updated timestamps when they expire.
Therefore, only call ->dirty_inode when I_DIRTY_INODE is set.
Based on a patch from Christoph Hellwig:
https://lore.kernel.org/r/20200325122825.1086872-4-hch@lst.de
Signed-off-by: Eric Biggers <redacted>
---
fs/ext4/inode.c | 12 +-----------
fs/f2fs/super.c | 3 ---
fs/fs-writeback.c | 6 +++---
fs/gfs2/super.c | 2 --
4 files changed, 4 insertions(+), 19 deletions(-)
From: Jan Kara <jack@suse.cz> Date: 2021-01-07 13:14:12
On Mon 04-01-21 16:54:43, Eric Biggers wrote:
From: Eric Biggers <redacted>
As was done for generic_update_time(), only pass I_DIRTY_TIME to
__mark_inode_dirty() when the inode's timestamps were actually updated
and lazytime is enabled. This avoids a weird edge case where
I_DIRTY_TIME could be set in i_state when lazytime isn't enabled.
Signed-off-by: Eric Biggers <redacted>
From: Jan Kara <jack@suse.cz> Date: 2021-01-07 13:18:36
On Mon 04-01-21 16:54:44, Eric Biggers wrote:
From: Eric Biggers <redacted>
There is no need to call ->dirty_inode for lazytime timestamp updates
(i.e. for __mark_inode_dirty(I_DIRTY_TIME)), since by the definition of
lazytime, filesystems must ignore these updates. Filesystems only need
to care about the updated timestamps when they expire.
Therefore, only call ->dirty_inode when I_DIRTY_INODE is set.
Based on a patch from Christoph Hellwig:
https://lore.kernel.org/r/20200325122825.1086872-4-hch@lst.de
Signed-off-by: Eric Biggers <redacted>
@@ -2264,16 +2264,16 @@ void __mark_inode_dirty(struct inode *inode, int flags)*Don'tdothisforI_DIRTY_PAGES-thatdoesn'tactually*dirtytheinodeitself*/-if(flags&(I_DIRTY_INODE|I_DIRTY_TIME)){+if(flags&I_DIRTY_INODE){trace_writeback_dirty_inode_start(inode,flags);if(sb->s_op->dirty_inode)sb->s_op->dirty_inode(inode,flags);
OK, but shouldn't we pass just (flags & I_DIRTY_INODE) to ->dirty_inode().
Just to make it clear what the filesystem is supposed to consume in
'flags'...
Honza
--
Jan Kara [off-list ref]
SUSE Labs, CR
From: Jan Kara <jack@suse.cz> Date: 2021-01-07 13:19:37
On Thu 07-01-21 14:17:53, Jan Kara wrote:
On Mon 04-01-21 16:54:44, Eric Biggers wrote:
quoted
From: Eric Biggers <redacted>
There is no need to call ->dirty_inode for lazytime timestamp updates
(i.e. for __mark_inode_dirty(I_DIRTY_TIME)), since by the definition of
lazytime, filesystems must ignore these updates. Filesystems only need
to care about the updated timestamps when they expire.
Therefore, only call ->dirty_inode when I_DIRTY_INODE is set.
Based on a patch from Christoph Hellwig:
https://lore.kernel.org/r/20200325122825.1086872-4-hch@lst.de
Signed-off-by: Eric Biggers <redacted>
@@ -2264,16 +2264,16 @@ void __mark_inode_dirty(struct inode *inode, int flags)*Don'tdothisforI_DIRTY_PAGES-thatdoesn'tactually*dirtytheinodeitself*/-if(flags&(I_DIRTY_INODE|I_DIRTY_TIME)){+if(flags&I_DIRTY_INODE){trace_writeback_dirty_inode_start(inode,flags);if(sb->s_op->dirty_inode)sb->s_op->dirty_inode(inode,flags);
OK, but shouldn't we pass just (flags & I_DIRTY_INODE) to ->dirty_inode().
Just to make it clear what the filesystem is supposed to consume in
'flags'...
Aha, you just did that in the following patch ;) So taking back my comment.
Honza
--
Jan Kara [off-list ref]
SUSE Labs, CR
From: Jan Kara <jack@suse.cz> Date: 2021-01-07 13:25:11
On Mon 04-01-21 16:54:47, Eric Biggers wrote:
quoted hunk
From: Eric Biggers <redacted>
Since I_DIRTY_TIME and I_DIRTY_INODE are mutually exclusive in i_state,
there's no need to check for I_DIRTY_TIME && !I_DIRTY_INODE. Just check
for I_DIRTY_TIME.
Signed-off-by: Eric Biggers <redacted>
---
fs/ext4/inode.c | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)
From: Jan Kara <jack@suse.cz> Date: 2021-01-07 14:03:27
On Mon 04-01-21 16:54:50, Eric Biggers wrote:
From: Eric Biggers <redacted>
Add a lazytime_expired method to 'struct super_operations'. Filesystems
can implement this to be notified when an inode's lazytime timestamps
have expired and need to be written to disk.
This avoids any potential ambiguity with
->dirty_inode(inode, I_DIRTY_SYNC), which can also mean a generic
dirtying of the inode, not just a lazytime timestamp expiration.
In particular, this will be useful for XFS.
If not implemented, then ->dirty_inode(inode, I_DIRTY_SYNC) continues to
be called.
Note that there are three cases where we have to make sure to call
lazytime_expired():
- __writeback_single_inode(): inode is being written now
- vfs_fsync_range(): inode is going to be synced
- iput(): inode is going to be evicted
In the latter two cases, the inode still needs to be put on the
writeback list. So, we can't just replace the calls to
mark_inode_dirty_sync() with lazytime_expired(). Instead, add a new
flag I_DIRTY_TIME_EXPIRED which can be passed to __mark_inode_dirty().
It's like I_DIRTY_SYNC, except it causes the filesystem to be notified
of a lazytime expiration rather than a generic I_DIRTY_SYNC.
Signed-off-by: Eric Biggers <redacted>
Hum, seeing this patch I kind of wonder: Why don't we dirty the inode after
expiring the lazytime timestamps with I_DIRTY_SYNC | I_DIRTY_TIME_EXPIRED
and propagate I_DIRTY_TIME_EXPIRED even to ->dirty_inode() where XFS can
catch it and act? Functionally it would be the same but we'd save a bunch
of generic code and ->lazytime_expired helper used just by a single
filesystem...
Honza
@@ -231,6 +231,7 @@ filesystem. As of kernel 2.6.22, the following members are defined: void (*destroy_inode)(struct inode *); void (*dirty_inode) (struct inode *, int flags);+ void (*lazytime_expired) (struct inode *); int (*write_inode) (struct inode *, int); void (*drop_inode) (struct inode *); void (*delete_inode) (struct inode *);
@@ -275,6 +276,15 @@ or bottom half). not its data. If the update needs to be persisted by fdatasync(), then I_DIRTY_DATASYNC will be set in the flags argument.+``lazytime_expired``+ when the lazytime mount option is enabled, this method is+ called when an inode's in-memory updated timestamps have+ expired and thus need to be written to disk. This happens+ when the timestamps have been in memory for too long, when the+ inode is going to be evicted, or when userspace triggers a+ sync. If this method is not implemented, then+ ->dirty_inode(inode, I_DIRTY_SYNC) is called instead.+``write_inode`` this method is called when the VFS needs to write an inode to disc. The second parameter indicates whether the write should
From: Jan Kara <jack@suse.cz> Date: 2021-01-07 14:47:53
On Mon 04-01-21 16:54:40, Eric Biggers wrote:
From: Eric Biggers <redacted>
When lazytime is enabled and an inode with dirty timestamps is being
expired, either due to dirtytime_expire_interval being exceeded or due
to a sync or syncfs system call, we need to inform the filesystem that
the inode is dirty so that the inode's timestamps can be copied out to
the on-disk data structures. That's because if the filesystem supports
lazytime, it will have ignored the ->dirty_inode(inode, I_DIRTY_TIME)
notification when the timestamp was modified in memory.
Currently this is accomplished by calling mark_inode_dirty_sync() from
__writeback_single_inode(). However, this has the unfortunate side
effect of also putting the inode the writeback list. That's not
appropriate in this case, since the inode is already being written.
That causes the inode to remain dirty after a sync. Normally that's
just wasteful, as it causes the inode to be written twice. But when
fscrypt is used this bug also partially breaks the
FS_IOC_REMOVE_ENCRYPTION_KEY ioctl, as the ioctl reports that files are
still in-use when they aren't. For more details, see the original
report at https://lore.kernel.org/r/20200306004555.GB225345@gmail.com
Fix this by calling ->dirty_inode(inode, I_DIRTY_SYNC) directly instead
of mark_inode_dirty_sync().
This fixes xfstest generic/580 when lazytime is enabled.
A later patch will introduce a ->lazytime_expired method to cleanly
separate out the lazytime expiration case, in particular for XFS which
uses the VFS-level dirtiness tracking only for lazytime. But that's
separate from fixing this bug. Also, note that XFS will incorrectly
ignore the I_DIRTY_SYNC notification from __writeback_single_inode()
both before and after this patch, as I_DIRTY_TIME was already cleared in
i_state. Later patches will fix this separate bug.
Fixes: 0ae45f63d4ef ("vfs: add support for a lazytime mount option")
Cc: stable@vger.kernel.org
Signed-off-by: Eric Biggers <redacted>
Good catch! It could also cause issues with filesystem freezing which kind
of assumes that the filesystem will be clean after sync_filesystem()
(otherwise writeback threads can get stalled on frozen filesystem while
holding some locks and generally the system behavior becomes kind of
awkward).
@@ -1509,11 +1509,22 @@ __writeback_single_inode(struct inode *inode, struct writeback_control *wbc)spin_unlock(&inode->i_lock);-if(dirty&I_DIRTY_TIME)-mark_inode_dirty_sync(inode);/* Don't write the inode if only I_DIRTY_PAGES was set */if(dirty&~I_DIRTY_PAGES){-interr=write_inode(inode,wbc);+interr;++/*+*Iftheinodeisbeingwrittenduetoalazytimetimestamp+*expiration,thenthefilesystemneedstobenotifiedaboutit+*sothate.g.thefilesystemcanupdateon-diskfieldsand+*journalthetimestampupdate.Justcallingwrite_inode()+*isn'tenough.Don'tcallmark_inode_dirty_sync(),asthat+*wouldputtheinodebackonthedirtylist.+*/+if((dirty&I_DIRTY_TIME)&&inode->i_sb->s_op->dirty_inode)+inode->i_sb->s_op->dirty_inode(inode,I_DIRTY_SYNC);++err=write_inode(inode,wbc);if(ret==0)ret=err;}
I have to say I dislike this special call of ->dirty_inode(). It works but
it makes me wonder, didn't we forget about something or won't we forget in
the future? Because it's very easy to miss this special case...
I think attached patch (compile-tested only) should actually fix the
problem as well without this special ->dirty_inode() call. It basically
only moves the mark_inode_dirty_sync() before inode->i_state clearing.
Because conceptually mark_inode_dirty_sync() is IMO the right function to
call. It will take care of clearing I_DIRTY_TIME flag (because we are
setting I_DIRTY_SYNC), it will also not touch inode->i_io_list if the inode
is queued for sync (I_SYNC_QUEUED is set in that case). The only problem
with calling it was that it was called *after* clearing dirty bits from
i_state... What do you think?
Honza
--
Jan Kara [off-list ref]
SUSE Labs, CR
From: Eric Biggers <ebiggers@kernel.org> Date: 2021-01-07 19:07:34
On Thu, Jan 07, 2021 at 02:24:12PM +0100, Jan Kara wrote:
On Mon 04-01-21 16:54:47, Eric Biggers wrote:
quoted
From: Eric Biggers <redacted>
Since I_DIRTY_TIME and I_DIRTY_INODE are mutually exclusive in i_state,
there's no need to check for I_DIRTY_TIME && !I_DIRTY_INODE. Just check
for I_DIRTY_TIME.
Signed-off-by: Eric Biggers <redacted>
---
fs/ext4/inode.c | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)
From: Eric Biggers <ebiggers@kernel.org> Date: 2021-01-07 19:11:06
On Thu, Jan 07, 2021 at 02:13:28PM +0100, Jan Kara wrote:
On Mon 04-01-21 16:54:43, Eric Biggers wrote:
quoted
From: Eric Biggers <redacted>
As was done for generic_update_time(), only pass I_DIRTY_TIME to
__mark_inode_dirty() when the inode's timestamps were actually updated
and lazytime is enabled. This avoids a weird edge case where
I_DIRTY_TIME could be set in i_state when lazytime isn't enabled.
Signed-off-by: Eric Biggers <redacted>
@@ -1509,11 +1509,22 @@ __writeback_single_inode(struct inode *inode, struct writeback_control *wbc)spin_unlock(&inode->i_lock);-if(dirty&I_DIRTY_TIME)-mark_inode_dirty_sync(inode);/* Don't write the inode if only I_DIRTY_PAGES was set */if(dirty&~I_DIRTY_PAGES){-interr=write_inode(inode,wbc);+interr;++/*+*Iftheinodeisbeingwrittenduetoalazytimetimestamp+*expiration,thenthefilesystemneedstobenotifiedaboutit+*sothate.g.thefilesystemcanupdateon-diskfieldsand+*journalthetimestampupdate.Justcallingwrite_inode()+*isn'tenough.Don'tcallmark_inode_dirty_sync(),asthat+*wouldputtheinodebackonthedirtylist.+*/+if((dirty&I_DIRTY_TIME)&&inode->i_sb->s_op->dirty_inode)+inode->i_sb->s_op->dirty_inode(inode,I_DIRTY_SYNC);++err=write_inode(inode,wbc);if(ret==0)ret=err;}
I have to say I dislike this special call of ->dirty_inode(). It works but
it makes me wonder, didn't we forget about something or won't we forget in
the future? Because it's very easy to miss this special case...
I think attached patch (compile-tested only) should actually fix the
problem as well without this special ->dirty_inode() call. It basically
only moves the mark_inode_dirty_sync() before inode->i_state clearing.
Because conceptually mark_inode_dirty_sync() is IMO the right function to
call. It will take care of clearing I_DIRTY_TIME flag (because we are
setting I_DIRTY_SYNC), it will also not touch inode->i_io_list if the inode
is queued for sync (I_SYNC_QUEUED is set in that case). The only problem
with calling it was that it was called *after* clearing dirty bits from
i_state... What do you think?
Honza
--
Jan Kara [off-list ref]
SUSE Labs, CR
quoted hunk
From 80ccc6a78d1c0532f600b98884f7a64e58333485 Mon Sep 17 00:00:00 2001
From: Jan Kara <jack@suse.cz>
Date: Thu, 7 Jan 2021 15:36:05 +0100
Subject: [PATCH] fs: Make sure inode is clean after __writeback_single_inode()
Signed-off-by: Jan Kara <jack@suse.cz>
---
fs/fs-writeback.c | 23 ++++++++++++-----------
1 file changed, 12 insertions(+), 11 deletions(-)
@@ -1509,8 +1512,6 @@ __writeback_single_inode(struct inode *inode, struct writeback_control *wbc)spin_unlock(&inode->i_lock);-if(dirty&I_DIRTY_TIME)-mark_inode_dirty_sync(inode);/* Don't write the inode if only I_DIRTY_PAGES was set */if(dirty&~I_DIRTY_PAGES){interr=write_inode(inode,wbc);
It looks like that's going to work, and it fixes the XFS bug too.
Note that if __writeback_single_inode() is called from writeback_single_inode()
(rather than writeback_sb_inodes()), then the inode might not be queued for
sync, in which case mark_inode_dirty_sync() will move it to a writeback list.
That's okay because afterwards, writeback_single_inode() will delete the inode
from any writeback list if it's been fully cleaned, right? So clean inodes
won't get left on a writeback list.
It's confusing because there are comments in writeback_single_inode() and above
__writeback_single_inode() that say that the inode must not be moved between
writeback lists. I take it that those comments are outdated, as they predate
I_SYNC_QUEUED being introduced by commit 5afced3bf281 ("writeback: Avoid
skipping inode writeback")?
- Eric
From: Eric Biggers <ebiggers@kernel.org> Date: 2021-01-07 22:06:41
On Thu, Jan 07, 2021 at 03:02:28PM +0100, Jan Kara wrote:
On Mon 04-01-21 16:54:50, Eric Biggers wrote:
quoted
From: Eric Biggers <redacted>
Add a lazytime_expired method to 'struct super_operations'. Filesystems
can implement this to be notified when an inode's lazytime timestamps
have expired and need to be written to disk.
This avoids any potential ambiguity with
->dirty_inode(inode, I_DIRTY_SYNC), which can also mean a generic
dirtying of the inode, not just a lazytime timestamp expiration.
In particular, this will be useful for XFS.
If not implemented, then ->dirty_inode(inode, I_DIRTY_SYNC) continues to
be called.
Note that there are three cases where we have to make sure to call
lazytime_expired():
- __writeback_single_inode(): inode is being written now
- vfs_fsync_range(): inode is going to be synced
- iput(): inode is going to be evicted
In the latter two cases, the inode still needs to be put on the
writeback list. So, we can't just replace the calls to
mark_inode_dirty_sync() with lazytime_expired(). Instead, add a new
flag I_DIRTY_TIME_EXPIRED which can be passed to __mark_inode_dirty().
It's like I_DIRTY_SYNC, except it causes the filesystem to be notified
of a lazytime expiration rather than a generic I_DIRTY_SYNC.
Signed-off-by: Eric Biggers <redacted>
Hum, seeing this patch I kind of wonder: Why don't we dirty the inode after
expiring the lazytime timestamps with I_DIRTY_SYNC | I_DIRTY_TIME_EXPIRED
and propagate I_DIRTY_TIME_EXPIRED even to ->dirty_inode() where XFS can
catch it and act? Functionally it would be the same but we'd save a bunch
of generic code and ->lazytime_expired helper used just by a single
filesystem...
Yes, that would be equivalent to what this patch does.
Either way, note that if we also use your suggestion for patch #1, then that
already fixes the XFS bug, since i_state will start containing I_DIRTY_TIME when
->dirty_inode(I_DIRTY_SYNC) is called. So xfs_fs_dirty_inode() will start
working as intended.
That makes introducing ->lazytime_expired (or equivalently I_DIRTY_TIME_EXPIRED)
kind of useless since it wouldn't actually fix anything.
So I'm tempted to just drop it.
The XFS developers might have a different opinion though, as they were the ones
who requested it originally:
https://lore.kernel.org/r/20200312143445.GA19160@infradead.orghttps://lore.kernel.org/r/20200325092057.GA25483@infradead.orghttps://lore.kernel.org/r/20200325154759.GY29339@magnoliahttps://lore.kernel.org/r/20200312223913.GL10776@dread.disaster.area
Any thoughts from anyone about whether we should still introduce a separate
notification for lazytime expiration, vs. just using ->dirty_inode(I_DIRTY_SYNC)
with I_DIRTY_TIME in i_state?
- Eric
From: Christoph Hellwig <hch@lst.de> Date: 2021-01-08 08:55:32
On Thu, Jan 07, 2021 at 01:46:37PM -0800, Eric Biggers wrote:
It looks like that's going to work, and it fixes the XFS bug too.
Note that if __writeback_single_inode() is called from writeback_single_inode()
(rather than writeback_sb_inodes()), then the inode might not be queued for
sync, in which case mark_inode_dirty_sync() will move it to a writeback list.
That's okay because afterwards, writeback_single_inode() will delete the inode
from any writeback list if it's been fully cleaned, right? So clean inodes
won't get left on a writeback list.
It's confusing because there are comments in writeback_single_inode() and above
__writeback_single_inode() that say that the inode must not be moved between
writeback lists. I take it that those comments are outdated, as they predate
I_SYNC_QUEUED being introduced by commit 5afced3bf281 ("writeback: Avoid
skipping inode writeback")?
Yes. I think we need to update the comment as well.
From: Christoph Hellwig <hch@lst.de> Date: 2021-01-08 09:02:18
+ /*
+ * If inode has dirty timestamps and we need to write them, call
+ * mark_inode_dirty_sync() to notify filesystem about it.
+ */
+ if (inode->i_state & I_DIRTY_TIME &&
+ (wbc->for_sync || wbc->sync_mode == WB_SYNC_ALL ||
+ time_after(jiffies, inode->dirtied_time_when +
+ dirtytime_expire_interval * HZ))) {
If we're touching this area, it would be nice to split this condition
into a readable helper ala:
static inline bool inode_needs_timestamp_sync(struct writeback_control *wbc,
struct inode *inode)
{
if (!(inode->i_state & I_DIRTY_TIME))
return false;
if (wbc->for_sync || wbc->sync_mode == WB_SYNC_ALL)
return true;
return time_after(jiffies, inode->dirtied_time_when +
dirtytime_expire_interval * HZ);
}
I still find the way ->dirty_inode is used very confusing, but with this
series and Jan's first patch I think we have a good enough state for now
and don't need to add a method just for XFS. I still think it might make
sense to eventually revisit how file systems are notified about dirtying.
From: Christoph Hellwig <hch@lst.de> Date: 2021-01-08 09:16:08
On Mon, Jan 04, 2021 at 04:54:51PM -0800, Eric Biggers wrote:
From: Eric Biggers <redacted>
The comment in xfs_file_aio_write_checks() about calling file_modified()
after dropping the ilock doesn't make sense, because the code that
unconditionally acquires and drops the ilock was removed by
commit 467f78992a07 ("xfs: reduce ilock hold times in
xfs_file_aio_write_checks").
Remove this outdated comment.
Yes, this looks good, I actually have the removal included in a WIP
patch as well, but splitting it out like this look good to me:
Reviewed-by: Christoph Hellwig <hch@lst.de>
From: Matthew Wilcox <willy@infradead.org> Date: 2021-01-08 20:39:36
On Thu, Jan 07, 2021 at 03:47:09PM +0100, Jan Kara wrote:
I think attached patch (compile-tested only) should actually fix the
problem as well without this special ->dirty_inode() call. It basically
only moves the mark_inode_dirty_sync() before inode->i_state clearing.
Because conceptually mark_inode_dirty_sync() is IMO the right function to
call. It will take care of clearing I_DIRTY_TIME flag (because we are
setting I_DIRTY_SYNC), it will also not touch inode->i_io_list if the inode
is queued for sync (I_SYNC_QUEUED is set in that case). The only problem
with calling it was that it was called *after* clearing dirty bits from
i_state... What do you think?
From: Eric Biggers <ebiggers@kernel.org> Date: 2021-01-09 17:12:28
On Fri, Jan 08, 2021 at 10:01:33AM +0100, Christoph Hellwig wrote:
quoted
+ /*
+ * If inode has dirty timestamps and we need to write them, call
+ * mark_inode_dirty_sync() to notify filesystem about it.
+ */
+ if (inode->i_state & I_DIRTY_TIME &&
+ (wbc->for_sync || wbc->sync_mode == WB_SYNC_ALL ||
+ time_after(jiffies, inode->dirtied_time_when +
+ dirtytime_expire_interval * HZ))) {
If we're touching this area, it would be nice to split this condition
into a readable helper ala:
static inline bool inode_needs_timestamp_sync(struct writeback_control *wbc,
struct inode *inode)
{
if (!(inode->i_state & I_DIRTY_TIME))
return false;
if (wbc->for_sync || wbc->sync_mode == WB_SYNC_ALL)
return true;
return time_after(jiffies, inode->dirtied_time_when +
dirtytime_expire_interval * HZ);
}
I didn't end up doing this since it would only be called once, and IMO it's more
readable to keep it inlined next to the comment that explains what's going on.
Especially considering the later patch that drops the check for wbc->for_sync.
- Eric