[PATCHSET 0/2] dax: fix broken pmem poison narrative

Hi all,

Our current "advice" to people using persistent memory and FSDAX who
wish to recover upon receipt of a media error (aka 'hwpoison') event
from ACPI is to punch-hole that part of the file and then pwrite it,
which will magically cause the pmem to be reinitialized and the poison
to be cleared.

Punching doesn't make any sense at all -- we don't allow userspace to
allocate from specific parts of the storage, and another writer could
grab the poisoned range in the meantime.  In other words, the advice is
seriously overfitted to incidental xfs and ext4 behavior and can
completely fail.  Worse yet, that concurrent writer now has to deal with
the poison that it didn't know about, and someone else is trying to fix.

AFAICT, the only reason why the "punch and write" dance works at all is
that the XFS and ext4 currently call blkdev_issue_zeroout when
allocating pmem as part of a pwrite call.  A pwrite without the punch
won't clear the poison, because pwrite on a DAX file calls
dax_direct_access to access the memory directly, and dax_direct_access
is only smart enough to bail out on poisoned pmem.  It does not know how
to clear it.  Userspace could solve the problem by calling FIEMAP and
issuing a BLKZEROOUT, but that requires rawio capabilities.

The whole pmem poison recovery story is is wrong and needs to be
corrected ASAP before everyone else starts doing this.  Therefore,
create a dax_zeroinit_range function that filesystems can call to reset
the contents of the pmem to a known value and clear any state associated
with the media error.  Then, connect FALLOC_FL_ZERO_RANGE to this new
function (for DAX files) so that unprivileged userspace has a safe way
to reset the pmem and clear media errors.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=dax-zeroinit-clear-poison-5.15
---
 fs/dax.c            |   72 +++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/ext4/extents.c   |   19 +++++++++++++
 fs/xfs/xfs_file.c   |   20 ++++++++++++++
 include/linux/dax.h |    7 +++++
 4 files changed, 118 insertions(+)

From: Darrick J. Wong <djwong@kernel.org>

Our current "advice" to people using persistent memory and FSDAX who
wish to recover upon receipt of a media error (aka 'hwpoison') event
from ACPI is to punch-hole that part of the file and then pwrite it,
which will magically cause the pmem to be reinitialized and the poison
to be cleared.

Punching doesn't make any sense at all -- the (re)allocation on pwrite
does not permit the caller to specify where to find blocks, which means
that we might not get the same pmem back.  This pushes the user farther
away from the goal of reinitializing poisoned memory and leads to
complaints about unnecessary file fragmentation.

AFAICT, the only reason why the "punch and write" dance works at all is
that the XFS and ext4 currently call blkdev_issue_zeroout when
allocating pmem ahead of a write call.  Even a regular overwrite won't
clear the poison, because dax_direct_access is smart enough to bail out
on poisoned pmem, but not smart enough to clear it.  To be fair, that
function maps pages and has no idea what kinds of reads and writes the
caller might want to perform.

Therefore, create a dax_zeroinit_range function that filesystems can
call from fallocate ZERO RANGE requests to reset the pmem contents to
zero and clear hardware media error flags, and hook it up to XFS.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 fs/dax.c            |   72 +++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_file.c   |   20 ++++++++++++++
 include/linux/dax.h |    7 +++++
 3 files changed, 99 insertions(+)

diff --git a/fs/dax.c b/fs/dax.c
index da41f9363568..fdd7b94f34f0 100644
--- a/fs/dax.c
+++ b/fs/dax.c

@@ -1742,3 +1742,75 @@ vm_fault_t dax_finish_sync_fault(struct vm_fault *vmf,
 	return dax_insert_pfn_mkwrite(vmf, pfn, order);
 }
 EXPORT_SYMBOL_GPL(dax_finish_sync_fault);
+
+static loff_t
+dax_zeroinit_actor(struct inode *inode, loff_t pos, loff_t length, void *data,
+		struct iomap *iomap, struct iomap *srcmap)
+{
+	sector_t sector = iomap_sector(iomap, pos);
+	int ret;
+
+	if (!iomap->bdev)
+		return -ECANCELED;
+
+	/* Must be able to zero storage directly without fs intervention. */
+	if (iomap->flags & IOMAP_F_SHARED)
+		return -ECANCELED;
+	if (srcmap != iomap)
+		return -ECANCELED;
+
+	switch (iomap->type) {
+	case IOMAP_MAPPED:
+		ret = blkdev_issue_zeroout(iomap->bdev, sector,
+				length >> SECTOR_SHIFT, GFP_KERNEL, 0);
+		if (ret)
+			return ret;
+		fallthrough;
+	case IOMAP_UNWRITTEN:
+		return length;
+	}
+
+	/* Reject holes, inline data, or delalloc extents. */
+	return -ECANCELED;
+}
+
+/*
+ * Initialize storage mapped to a DAX-mode file to a known value and ensure the
+ * media are ready to accept read and write commands.  This requires the use of
+ * the block layer's zeroout function (with zero-page fallback enabled) to
+ * write zeroes to a pmem region and to reset any hardware media error state.
+ *
+ * The range arguments must be aligned to sector size.  The file must be backed
+ * by a block device.  The extents returned must not require copy on write (or
+ * any other mapping interventions from the filesystem) and must be contiguous.
+ * @done will be set to true if the reset succeeded.
+ */
+int
+dax_zeroinit_range(struct inode *inode, loff_t pos, loff_t len, bool *done,
+		const struct iomap_ops *ops)
+{
+	loff_t ret;
+
+	if (!IS_DAX(inode))
+		return -EINVAL;
+	if ((pos | len) & (SECTOR_SIZE - 1))
+		return -EINVAL;
+	if (pos + len > i_size_read(inode))
+		return -EINVAL;
+
+	while (len > 0) {
+		ret = iomap_apply(inode, pos, len, IOMAP_REPORT, ops, NULL,
+				dax_zeroinit_actor);
+		if (ret == -ECANCELED)
+			return 0;
+		if (ret < 0)
+			return ret;
+
+		pos += ret;
+		len -= ret;
+	}
+
+	*done = true;
+	return 0;
+}
+EXPORT_SYMBOL_GPL(dax_zeroinit_range);

diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
index cc3cfb12df53..e77793820cf3 100644
--- a/fs/xfs/xfs_file.c
+++ b/fs/xfs/xfs_file.c

@@ -956,6 +956,25 @@ xfs_file_fallocate(
 			goto out_unlock;
 	}
 
+	/*
+	 * If the file is in DAX mode, try to use a DAX-specific function to
+	 * zero the region.  We can fall back to punch-and-realloc if necessary.
+	 */
+	if ((mode & FALLOC_FL_ZERO_RANGE) && IS_DAX(inode)) {
+		bool	did_zeroout = false;
+
+		trace_xfs_zero_file_space(ip);
+
+		error = dax_zeroinit_range(inode, offset, len, &did_zeroout,
+				&xfs_read_iomap_ops);
+		if (error == -EINVAL)
+			error = 0;
+		if (error)
+			goto out_unlock;
+		if (did_zeroout)
+			goto done;
+	}
+
 	if (mode & FALLOC_FL_PUNCH_HOLE) {
 		error = xfs_free_file_space(ip, offset, len);
 		if (error)

@@ -1059,6 +1078,7 @@ xfs_file_fallocate(
 		}
 	}
 
+done:
 	if (file->f_flags & O_DSYNC)
 		flags |= XFS_PREALLOC_SYNC;
 

diff --git a/include/linux/dax.h b/include/linux/dax.h
index b52f084aa643..df52d0ce0ee0 100644
--- a/include/linux/dax.h
+++ b/include/linux/dax.h

@@ -152,6 +152,8 @@ struct page *dax_layout_busy_page(struct address_space *mapping);
 struct page *dax_layout_busy_page_range(struct address_space *mapping, loff_t start, loff_t end);
 dax_entry_t dax_lock_page(struct page *page);
 void dax_unlock_page(struct page *page, dax_entry_t cookie);
+int dax_zeroinit_range(struct inode *inode, loff_t pos, loff_t len, bool *done,
+			const struct iomap_ops *ops);
 #else
 static inline bool bdev_dax_supported(struct block_device *bdev,
 		int blocksize)

@@ -201,6 +203,11 @@ static inline dax_entry_t dax_lock_page(struct page *page)
 static inline void dax_unlock_page(struct page *page, dax_entry_t cookie)
 {
 }
+static inline int dax_zeroinit_range(struct inode *inode, loff_t pos, loff_t len,
+		bool *done, const struct iomap_ops *ops)
+{
+	return 0;
+}
 #endif
 
 #if IS_ENABLED(CONFIG_DAX)

From: Darrick J. Wong <djwong@kernel.org>

Our current "advice" to people using persistent memory and FSDAX who
wish to recover upon receipt of a media error (aka 'hwpoison') event
from ACPI is to punch-hole that part of the file and then pwrite it,
which will magically cause the pmem to be reinitialized and the poison
to be cleared.

Punching doesn't make any sense at all -- the (re)allocation on pwrite
does not permit the caller to specify where to find blocks, which means
that we might not get the same pmem back.  This pushes the user farther
away from the goal of reinitializing poisoned memory and leads to
complaints about unnecessary file fragmentation.

Now that we've created a dax_zeroinit_range to perform what amounts to a
low level format of pmem, hook it up to ext4.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 fs/ext4/extents.c |   19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
index 92ad64b89d9b..630ce5338368 100644
--- a/fs/ext4/extents.c
+++ b/fs/ext4/extents.c

@@ -29,6 +29,7 @@
 #include <linux/fiemap.h>
 #include <linux/backing-dev.h>
 #include <linux/iomap.h>
+#include <linux/dax.h>
 #include "ext4_jbd2.h"
 #include "ext4_extents.h"
 #include "xattr.h"

@@ -4678,6 +4679,24 @@ long ext4_fallocate(struct file *file, int mode, loff_t offset, loff_t len)
 	}
 
 	if (mode & FALLOC_FL_ZERO_RANGE) {
+		/*
+		 * If the file is in DAX mode, try to use a DAX-specific
+		 * function to zero the region.
+		 */
+		if (IS_DAX(inode)) {
+			bool	did_zeroout = false;
+
+			inode_lock(inode);
+
+			ret = dax_zeroinit_range(inode, offset, len,
+					&did_zeroout, &ext4_iomap_report_ops);
+			if (ret == -EINVAL)
+				ret = 0;
+			if (ret || did_zeroout)
+				goto out;
+
+			inode_unlock(inode);
+		}
 		ret = ext4_zero_range(file, offset, len, mode);
 		goto exit;
 	}

On Mon, Aug 16, 2021 at 02:05:18PM -0700, Darrick J. Wong wrote:

Hi all,

Our current "advice" to people using persistent memory and FSDAX who
wish to recover upon receipt of a media error (aka 'hwpoison') event
from ACPI is to punch-hole that part of the file and then pwrite it,
which will magically cause the pmem to be reinitialized and the poison
to be cleared.

Punching doesn't make any sense at all -- we don't allow userspace to
allocate from specific parts of the storage, and another writer could
grab the poisoned range in the meantime.  In other words, the advice is
seriously overfitted to incidental xfs and ext4 behavior and can
completely fail.  Worse yet, that concurrent writer now has to deal with
the poison that it didn't know about, and someone else is trying to fix.

AFAICT, the only reason why the "punch and write" dance works at all is
that the XFS and ext4 currently call blkdev_issue_zeroout when
allocating pmem as part of a pwrite call.  A pwrite without the punch
won't clear the poison, because pwrite on a DAX file calls
dax_direct_access to access the memory directly, and dax_direct_access
is only smart enough to bail out on poisoned pmem.  It does not know how
to clear it.  Userspace could solve the problem by calling FIEMAP and
issuing a BLKZEROOUT, but that requires rawio capabilities.

The whole pmem poison recovery story is is wrong and needs to be
corrected ASAP before everyone else starts doing this.  Therefore,
create a dax_zeroinit_range function that filesystems can call to reset
the contents of the pmem to a known value and clear any state associated
with the media error.  Then, connect FALLOC_FL_ZERO_RANGE to this new
function (for DAX files) so that unprivileged userspace has a safe way
to reset the pmem and clear media errors.

This is a sample copy of a SIGBUS handler that will dump out the siginfo
data, call ZERO_RANGE to clear the poison, and then simulates being
fortunate enough to be able to reconstruct the file contents from
scratch.

Note that I haven't tested this even with simulated pmem because I
cannot figure out how to inject a poison error into the pmem in such a
way that the nvdimm driver records it in the badblocks table.
madvise(HWPOISON) calls the SIGBUS handler, but that code path never
goes outside of the memory manager.

int fd = open(...);
char *data = mmap(fd, ... MAP_SYNC);

static void handle_sigbus(int signal, siginfo_t *info, void *dontcare)
{
	char *buf;
	loff_t err_offset = (char *)info->si_addr - data;
	loff_t err_len = (1ULL << info->si_addr_lsb);
	ssize_t ret;

	printf("RECEIVED SIGBUS (POISON HANDLER)!\n");
	printf("    signal %d\n", info->si_signo);
	printf("    errno %d\n", info->si_errno);
	printf("    addr %p\n", info->si_addr);
	printf("    addr_lsb %d\n", info->si_addr_lsb);

	if (info->si_signo != SIGBUS) {
		printf("    code 0x%x\n", info->si_code);
		return;
	}

	switch (info->si_code) {
	case BUS_ADRALN:
		printf("    code: BUS_ADRALN\n");
		break;
	case BUS_ADRERR:
		printf("    code: BUS_ADRERR\n");
		break;
	case BUS_OBJERR:
		printf("    code: BUS_OBJERR\n");
		break;
	case BUS_MCEERR_AR:
		printf("    code: BUS_MCEERR_AR\n");
		break;
	case BUS_MCEERR_AO:
		printf("    code: BUS_MCEERR_AO\n");
		break;
	default:
		printf("    code 0x%x\n", info->si_code);
		break;
	}

	printf("    err_offset %lld\n", (unsigned long long)err_offset);
	printf("    err_len %lld\n", (unsigned long long)err_len);

	if (info->si_code != BUS_MCEERR_AR)
		return;

	/* clear poison and reset pmem to initial value */
	ret = fallocate(fd, FALLOC_FL_ZERO_RANGE, err_offset, err_len);
	if (ret) {
		perror("fallocate");
		exit(9);
	}

	/* simulate being lucky enough to be able to reconstruct the data */
	buf = malloc(err_len);
	if (!buf) {
		perror("malloc pwrite buf");
		exit(10);
	}

	memset(buf, 0x59, err_len);

	ret = pwrite(fd, buf, err_len, err_offset);
	if (ret < 0) {
		perror("pwrite");
		exit(11);
	}
	if (ret != err_len) {
		fprintf(stderr, "short write %zd bytes, wanted %lld\n",
				ret, (long long)err_len);
		exit(12);
	}

	free(buf);
}

--D

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=dax-zeroinit-clear-poison-5.15
---
 fs/dax.c            |   72 +++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/ext4/extents.c   |   19 +++++++++++++
 fs/xfs/xfs_file.c   |   20 ++++++++++++++
 include/linux/dax.h |    7 +++++
 4 files changed, 118 insertions(+)

On Mon, Aug 16, 2021 at 02:05:18PM -0700, Darrick J. Wong wrote:

AFAICT, the only reason why the "punch and write" dance works at all is
that the XFS and ext4 currently call blkdev_issue_zeroout when
allocating pmem as part of a pwrite call.  A pwrite without the punch
won't clear the poison, because pwrite on a DAX file calls
dax_direct_access to access the memory directly, and dax_direct_access
is only smart enough to bail out on poisoned pmem.  It does not know how
to clear it.  Userspace could solve the problem by calling FIEMAP and
issuing a BLKZEROOUT, but that requires rawio capabilities.

The whole pmem poison recovery story is is wrong and needs to be
corrected ASAP before everyone else starts doing this.  Therefore,
create a dax_zeroinit_range function that filesystems can call to reset
the contents of the pmem to a known value and clear any state associated
with the media error.  Then, connect FALLOC_FL_ZERO_RANGE to this new
function (for DAX files) so that unprivileged userspace has a safe way
to reset the pmem and clear media errors.

I agree with the problem statement, but I don't think the fix is
significantly better than what we have, as it still magically overloads
other behavior.  I'd rather have an explicit operation to clear the
poison both at the syscall level (maybe another falloc mode), and the
internal kernel API level (new method in dax_operations).

Also for the next iteration please split the iomap changes from the
usage in xfs.

On Tue, Aug 17, 2021 at 08:39:00AM +0100, Christoph Hellwig wrote:

On Mon, Aug 16, 2021 at 02:05:18PM -0700, Darrick J. Wong wrote:

quoted

AFAICT, the only reason why the "punch and write" dance works at all is
that the XFS and ext4 currently call blkdev_issue_zeroout when
allocating pmem as part of a pwrite call.  A pwrite without the punch
won't clear the poison, because pwrite on a DAX file calls
dax_direct_access to access the memory directly, and dax_direct_access
is only smart enough to bail out on poisoned pmem.  It does not know how
to clear it.  Userspace could solve the problem by calling FIEMAP and
issuing a BLKZEROOUT, but that requires rawio capabilities.

The whole pmem poison recovery story is is wrong and needs to be
corrected ASAP before everyone else starts doing this.  Therefore,
create a dax_zeroinit_range function that filesystems can call to reset
the contents of the pmem to a known value and clear any state associated
with the media error.  Then, connect FALLOC_FL_ZERO_RANGE to this new
function (for DAX files) so that unprivileged userspace has a safe way
to reset the pmem and clear media errors.

I agree with the problem statement, but I don't think the fix is
significantly better than what we have, as it still magically overloads
other behavior.  I'd rather have an explicit operation to clear the
poison both at the syscall level (maybe another falloc mode), and the
internal kernel API level (new method in dax_operations).

I've long wondered why we can't just pass a write flag to the
direct_access functions so that pmem_dax_direct_access can clear the
poison.  Then we ought to be able to tell userspace that they can
recover from write errors by pwrite() or triggering a write fault on the
page, I think.  That's how userspace recovers from IO errors on
traditional disks; I've never understood why it has to be any different
now.

Also for the next iteration please split the iomap changes from the
usage in xfs.

ok.

--D

On 8/17/2021 12:39 AM, Christoph Hellwig wrote:

On Mon, Aug 16, 2021 at 02:05:18PM -0700, Darrick J. Wong wrote:

quoted

AFAICT, the only reason why the "punch and write" dance works at all is
that the XFS and ext4 currently call blkdev_issue_zeroout when
allocating pmem as part of a pwrite call.  A pwrite without the punch
won't clear the poison, because pwrite on a DAX file calls
dax_direct_access to access the memory directly, and dax_direct_access
is only smart enough to bail out on poisoned pmem.  It does not know how
to clear it.  Userspace could solve the problem by calling FIEMAP and
issuing a BLKZEROOUT, but that requires rawio capabilities.

The whole pmem poison recovery story is is wrong and needs to be
corrected ASAP before everyone else starts doing this.  Therefore,
create a dax_zeroinit_range function that filesystems can call to reset
the contents of the pmem to a known value and clear any state associated
with the media error.  Then, connect FALLOC_FL_ZERO_RANGE to this new
function (for DAX files) so that unprivileged userspace has a safe way
to reset the pmem and clear media errors.

I agree with the problem statement, but I don't think the fix is
significantly better than what we have, as it still magically overloads

This fix guarantees contiguous pmem backend, versus the existing 
holepunch+pwrite ends up allocating an unused(by the filesystem)
pmem range from the pmem device, and the poison block stay poisoned,
not cleared by the existing method. This brings 3 problem: 1. the
backend become fragmented which negatively impacts the RDMA behavior,
2. what if the pmem device run out of extra clean blocks?  3. the user
interface that clears poison when device is unmounted is risky in
that it could damage filesystem metadata.

That said, another angle to view this patch is that, FALLOC_FL_ZERO_RANGE
is supposed to zero-range, for non-direct access block device, it's
okay to not actually write zeros to the media, but pmem device is
what-you-see-is-what-you-get, it seems making sense to actually writes
zero.

thanks,
-jane

other behavior.  I'd rather have an explicit operation to clear the
poison both at the syscall level (maybe another falloc mode), and the
internal kernel API level (new method in dax_operations).

Also for the next iteration please split the iomap changes from the
usage in xfs.