On 2021-07-19 15:07, Forza wrote:

On 2021-07-19 14:06, Forza wrote:

quoted

On 2021-07-13 09:37, Ulli Horlacher wrote:

quoted

On Mon 2021-07-12 (23:56), g.btrfs@cobb.uk.net wrote:

quoted

quoted

root@tsmsrvj:/etc# du -Hs /nfs/localhost/snapshots
du: WARNING: Circular directory structure.
This almost certainly means that you have a corrupted file system.
NOTIFY YOUR SYSTEM MANAGER.
The following directory is part of the cycle:
   /nfs/localhost/snapshots/spool

Sure. But it makes the useful operations work. du, find, ls -R, etc all
work properly on /nfs/localhost/fex.

Properly on /nfs/localhost/fex : yes
Properly on /nfs/localhost/snapshots : NO

And the error messages are annoying!

root@tsmsrvj:/etc# exportfs -v
/data/fex 
localhost.localdomain(rw,async,wdelay,crossmnt,no_root_squash,no_subtree_check,sec=sys,rw,secure,no_root_squash,no_all_squash) 

/data/snapshots 
localhost.localdomain(rw,async,wdelay,crossmnt,no_root_squash,no_subtree_check,sec=sys,rw,secure,no_root_squash,no_all_squash) 


root@tsmsrvj:/etc# mount -o vers=3 localhost:/data/fex 
/nfs/localhost/fex
root@tsmsrvj:/etc# mount -o vers=3 localhost:/data/snapshots 
/nfs/localhost/snapshots
root@tsmsrvj:/etc# mount | grep localhost
localhost:/data/fex on /nfs/localhost/fex type nfs 
(rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=127.0.0.1,mountvers=3,mountport=37961,mountproto=udp,local_lock=none,addr=127.0.0.1) 

localhost:/data/snapshots on /nfs/localhost/snapshots type nfs 
(rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=127.0.0.1,mountvers=3,mountport=37961,mountproto=udp,local_lock=none,addr=127.0.0.1) 

What kind of NFS server is this? Isn't UDP mounts legacy and not 
normally used by default?

Can you switch to an nfs4 server and try again? I also still think you 
should use fsid export option.

I'm replying to myself here because I booted up a VM with Fedora 34 and 
tested a similar setup as Mr Horlacher's and can reproduce the errors.

Setup:
1) create a subvolume /mnt/rootvol/nfs
2) create some snapshots:
btrfs sub snap /mnt/rootvol/nfs /mnt/rootvol/nfs/.snapshots/nfs-1
btrfs sub snap /mnt/rootvol/nfs /mnt/rootvol/nfs/.snapshots/nfs-2

3) export as:
/mnt/rootvol/nfs/ *(fsid=1234,no_root_squash)

4) mount -o vers=4 localhost:/mnt/rootvol/nfs /media/nfs-mnt/
5) "du -sh /media/nfs-mnt" fails with
"WARNING: Circular directory structure."

6) "ls -alR /mnt/nfs-mnt" fails with
"not listing already-listed directory"

In addition I have tried with various export options such as crossmnt, 
nohide and subtree_check. They do not improve the situation.

Also the behaviour is the same with nfs3 as with nfs4.

Full outputs are available at https://paste.ee/p/pkHLh

Perhaps the problem is that inode numbers are re-used inside snapshots 
and that nfsd doesn't understand how to handle this properly?

# ls -ila /media/nfs-mnt/
total 0
256 drwxr-xr-x. 1 root root 80 Jul 19 14:17 .
270 drwxr-xr-x. 1 root root 14 Jul 19 14:21 ..
259 -rw-r--r--. 1 root root  0 Jul 19 14:17 bar
261 -rw-r--r--. 1 root root  0 Jul 19 14:17 file1
262 -rw-r--r--. 1 root root  0 Jul 19 14:17 file2
263 -rw-r--r--. 1 root root  0 Jul 19 14:17 file3
258 -rw-r--r--. 1 root root  0 Jul 19 14:17 foo
257 drwxr-xr-x. 1 root root 30 Jul 19 15:02 .snapshots
260 -rw-r--r--. 1 root root  0 Jul 19 14:17 somefiles

# ls -ila /media/nfs-mnt/.snapshots/nfs-2/
total 0
256 drwxr-xr-x. 1 root root 80 Jul 19 14:17 .
257 drwxr-xr-x. 1 root root 30 Jul 19 15:02 ..
259 -rw-r--r--. 1 root root  0 Jul 19 14:17 bar
261 -rw-r--r--. 1 root root  0 Jul 19 14:17 file1
262 -rw-r--r--. 1 root root  0 Jul 19 14:17 file2
263 -rw-r--r--. 1 root root  0 Jul 19 14:17 file3
258 -rw-r--r--. 1 root root  0 Jul 19 14:17 foo
257 drwxr-xr-x. 1 root root 10 Jul 19 14:17 .snapshots
260 -rw-r--r--. 1 root root  0 Jul 19 14:17 somefiles


Using nfs4 exports and specifying each snapshot as its own fsid does not 
work either.

### /etc/exports
/mnt/rootvol/nfs/ *(fsid=root,no_root_squash,no_subtree_check)
/mnt/rootvol/nfs/.snapshots/nfs-1 
*(fsid=1000,no_root_squash,no_subtree_check)
/mnt/rootvol/nfs/.snapshots/nfs-2 
*(fsid=2000,no_root_squash,no_subtree_check)
/mnt/rootvol/nfs/.snapshots/nfs-3 
*(fsid=3000,no_root_squash,no_subtree_check)

# ls -laRi nfs-mnt/
nfs-mnt/:
total 0
256 drwxr-xr-x. 1 root root 80 Jul 19 14:17 .
270 drwxr-xr-x. 1 root root 14 Jul 19 14:21 ..
259 -rw-r--r--. 1 root root  0 Jul 19 14:17 bar
261 -rw-r--r--. 1 root root  0 Jul 19 14:17 file1
262 -rw-r--r--. 1 root root  0 Jul 19 14:17 file2
263 -rw-r--r--. 1 root root  0 Jul 19 14:17 file3
258 -rw-r--r--. 1 root root  0 Jul 19 14:17 foo
257 drwxr-xr-x. 1 root root 30 Jul 19 15:02 .snapshots
260 -rw-r--r--. 1 root root  0 Jul 19 14:17 somefiles

nfs-mnt/.snapshots:
total 0
257 drwxr-xr-x. 1 root root 30 Jul 19 15:02 .
256 drwxr-xr-x. 1 root root 80 Jul 19 14:17 ..
256 drwxr-xr-x. 1 root root 56 Jul 19 15:02 nfs-1
256 drwxr-xr-x. 1 root root 80 Jul 19 14:17 nfs-2
256 drwxr-xr-x. 1 root root 86 Jul 19 15:03 nfs-3
ls: nfs-mnt/.snapshots/nfs-1: not listing already-listed directory
ls: nfs-mnt/.snapshots/nfs-2: not listing already-listed directory
ls: nfs-mnt/.snapshots/nfs-3: not listing already-listed directory