On Thu, Feb 18, 2021 at 01:27:09PM -0800, Erik Jensen wrote:

On 2/18/21 4:15 AM, Matthew Wilcox wrote:

quoted

On Thu, Feb 18, 2021 at 04:54:46PM +0800, Qu Wenruo wrote:

quoted

Recently we got a strange bug report that, one 32bit systems like armv6
or non-64bit x86, certain large btrfs can't be mounted.

It turns out that, since page->index is just unsigned long, and on 32bit
systemts, that can just be 32bit.

And when filesystems is utilizing any page offset over 4T, page->index
get truncated, causing various problems.

4TB?  I think you mean 16TB (4kB * 4GB)

Yes, this is a known limitation.  Some vendors have gone to the trouble
of introducing a new page_index_t.  I'm not convinced this is a problem
worth solving.  There are very few 32-bit systems with this much storage
on a single partition (everything should work fine if you take a 20TB
drive and partition it into two 10TB partitions).

For what it's worth, I'm the reporter of the original bug. My use case is a
custom NAS system. It runs on a 32-bit ARM processor, and has 5 8TB drives,
which I'd like to use as a single, unified storage array. I chose btrfs for
this project due to the filesystem-integrated snapshots and checksums.
Currently, I'm working around this issue by exporting the raw drives using
nbd and mounting them on a 64-bit system to access the filesystem, but this
is very inconvenient, only allows one machine to access the filesystem at a
time, and prevents running any tools that need access to the filesystem
(such as backup and file sync utilities) on the NAS itself.

It sounds like this limitation would also prevent me from trying to use a
different filesystem on top of software RAID, since in that case the logical
filesystem would still be over 16TB.

quoted

As usual, the best solution is for people to stop buying 32-bit systems.

I purchased this device in 2018, so it's not exactly ancient. At the time,
it was the only SBC I could find that was low power, used ECC RAM, had a
crypto accelerator, and had multiple sata ports with port-multiplier
support.

I'm sorry you bought unsupported hardware.

This limitation has been known since at least 2009:
https://lore.kernel.org/lkml/19041.4714.686158.130252@notabene.brown/ (local)

In the last decade, nobody's tried to fix it in mainline that I know of.
As I said, some vendors have tried to fix it in their NAS products,
but I don't know where to find that patch any more.

https://bootlin.com/blog/large-page-support-for-nas-systems-on-32-bit-arm/
might help you, but btrfs might still contain assumptions that will trip
you up.