ENOSPC consequences on Linux 4.9

Hello,

I've observed an ENOSPC-related issue on a rather dated system. The
files are already copied to a newly created filesystem, and possibly
it's a bug that was fixed in newer versions, but I'm still curious to
find out what it was, and maybe there's a surviving bug involved.

The system:

# uname -a
Linux venus 4.9.0-14-amd64 #1 SMP Debian 4.9.246-2 (2020-12-17) x86_64 GNU/Linux
# btrfs --version
Btrfs v3.17
# cat /etc/debian_version 
9.9

The /home file system on sda4 ran out of space, some files were removed
from it, but it was still reporting not having space, and rebalancing
("btrfs balance start -dusage=5 /home") kept reporting "No space left on
device". Usually on that system it helps to add a device, rebalance it,
and remove the device in such a case; a loop device was added,
rebalancing still failed, and then the loop device was removed. That
happened between 11:58:35 and 12:01:17 in the attached kern.log
excerpts.

At about 13:50 in the same log I've joined and tried to add a 200 GB
device (sdb6) to rebalance with it, but that also failed with the same
error message. Attempted to remount with the clear_cache option, which
didn't seem to affect it, and neither did "btrfs balance start -dusage=0
/home". As was suggested on the btrfs IRC channel, tried to check
quotas, with `btrfs qgroup show /home` only printing "ERROR: can't
perform the search - No such file or directory" and "ERROR: can't list
qgroups: No such file or directory" (but I'm pretty sure that quotas
were never configured there). No snapshots or anything fancy was used on
it, and it was mounted with the default options.

Filesystem state at that point:

$ sudo btrfs fi df /home
Data, single: total=442.01GiB, used=338.97GiB
System, DUP: total=8.00MiB, used=80.00KiB
System, single: total=4.00MiB, used=0.00B
Metadata, DUP: total=11.81GiB, used=1.32GiB
GlobalReserve, single: total=512.00MiB, used=0.00B

$ sudo btrfs fi show /home
Label: none  uuid: f0d4f203-f70d-4a84-b18d-a1f34ffdaa8b
        Total devices 2 FS bytes used 340.29GiB
        devid    1 size 465.66GiB used 465.66GiB path /dev/sda4
        devid    2 size 200.00GiB used 8.00MiB path /dev/sdb6

At 15:10:22 I've created sdb7 with a new filesystem on it, and started
copying files with `rsync -a`. "Input/Output" errors started happening
soon, and rsync got stuck in a loop of the following two function calls,
as was seen with strace (a path to a file is replaced with "PATH"; it
was the same path in every call):

lstat("PATH", {st_mode=S_IFREG|0640, st_size=192512, ...}) = 0
getdents(3, /* 1 entries */, 32768)     = 56

Trying to access that path with ls(1) led to ls hanging as well, so I've
excluded it. As checked later, apparently the file was a few years
old. About half the files (by volume) still failed to be copied with
"Input/Output error", with mostly those "parent transid verify failed"
errors in dmesg, which was happening when trying to access them with ls
too. As the attached log points out, early in the copying process the
filesystem went read-only (15:11:19).

By 21:44 I've killed all the processes that were using /home, unmounted
it, mounted it read-only in a different location, and all the files were
readable again, without errors on reading or in dmesg.

The current filesystem state:

# btrfs fi df /mnt/old-home
Data, single: total=442.01GiB, used=338.97GiB
System, DUP: total=8.00MiB, used=80.00KiB
System, single: total=4.00MiB, used=0.00B
Metadata, RAID1: total=8.00MiB, used=4.73MiB
Metadata, DUP: total=11.81GiB, used=1.32GiB
GlobalReserve, single: total=512.00MiB, used=0.00B

# btrfs fi show /mnt/old-home
Label: none  uuid: f0d4f203-f70d-4a84-b18d-a1f34ffdaa8b
        Total devices 2 FS bytes used 340.29GiB
        devid    1 size 465.66GiB used 465.66GiB path /dev/sda4
        devid    2 size 200.00GiB used 8.00MiB path /dev/sdb6

Booting into a newer operating system or updating it now would be
problematic, but potentially dangerous/invasive to that filesystem
repairs and checks are fine, since it's not used or needed anymore. Is
it the time to run `btrfs check` or `btrfs rescue`?

The kern.log excerpts are attached.