[PATCH v2 00/20] btrfs-progs: Rework of "subvolume list/show" and relax the root privileges of them

STALE2979d

Revision v2 of 2 in this series.

26 messages, 2 authors, 2018-08-21 · open the first message on its own page

[PATCH v2 00/20] btrfs-progs: Rework of "subvolume list/show" and relax the root privileges of them

From: Misono Tomohiro <hidden>
Date: 2018-06-18 08:38:13

Changelog
 
 v1 -> v2: 
  generally update whole patch set, especially:
   - rebased to progs 4.17
   - Improve error handling
   - Update man/help/commit message
   - Add/Update several options of sub list:
      -f ... follow mounted subvolumes
      -a ... remove meaningless filter
      -A ... print path in absolute path
      --nosort ... output results incrementally
     Please see below examples
=====
github:  https://github.com/t-msn/btrfs-progs/tree/rework-sub-list

Hello,

This series requires some new ioctls which are now in kernel 4.18-rc1. 

The aim of this series is to relax the root privileges of "sub list/show"
while keeping as much output consistency between root and non-privileged
user. For "subvolume list", default output has been changed from current
btrfs-progs (in both old and new kernel) and some options are newly added.
For "subvolume show", root's output is the same as before but there are
some difference from non-privileged user's output. 

Please see below examples.


* Behavior summary of new "sub list"
  - default (no option)
    - lists subvolumes below the specified path (inc. path itself)
    - If new ioctls exists
      - the path can be non-subvolume directory
      - non-privileged user can call it
        (subvolumes to which the user cannot access will be skipped)

  - -f
    - follow mounted subvolume below the specified path and list them too 
      (only if it is the same filesystem)

  - -a
    - updated to remove filter. i.e. the output is the same as current progs
      without option (require root privileges)

  - -A
    - print path in absolute path

  -- nosort
    - output results incrementally without loading information to memory

 [Example]
  $ mkfs.btrfs -f $DEV
  $ mkfs.btrfs -f $DEV2
  $ mount $DEV $MNT

  $ btrfs subvolume create $MNT/AAA
  $ btrfs subvolume create $MNT/BBB
  $ btrfs subvolume create $MNT/CCC
  $ btrfs subvolume create $MNT/DDD
  $ mkdir $MNT/AAA/bbb
  $ mkdir $MNT/AAA/ccc
  $ mkdir $MNT/AAA/other

  $ umount $MNT
  $ mount -o subvol=AAA $DEV $MNT
  $ mount -o subvol=BBB $DEV $MNT/bbb
  $ mount -o subvol=CCC $DEV $MNT/ccc
  $ mount -o $DEV2 $MNT/other

  $ btrfs subvolume list $MNT # print subvolumes below the path
  ID 256 gen 10 top level 5 path .

  $ btrfs subvolume list -A $MNT # print path in absolute path
  ID 256 gen 10 top level 5 path /mnt

  $ btrfs subvolume list -f $MNT # follow mounted subvolumes too
  ID 256 gen 10 top level 5 path .
  ID 258 gen 7 top level 5 path bbb
  ID 259 gen 8 top level 5 path ccc

  $ btrfs subvolume list -a $MNT
  # print all subvolumes in the fs. same output as progs<=4.17 without option
  ID 256 gen 10 top level 5 path AAA
  ID 258 gen 7 top level 5 path BBB
  ID 259 gen 8 top level 5 path CCC
  ID 260 gen 9 top level 5 path DDD

 More details are in each commit log.


* Behavior summary of new "sub show"
  - No change for root's output
  - If new ioctls exists, non-privileged user can call it
    - In that case, path to be shown is absolute path
      (for root, it is relative to top-level subvolume)
      Also, snapshots to be shown are to which the user can
      access from current mount point.
      (for root, all snapshots in the fs)


* Patch structure
The first several patches update libbtrfsutil and the latter patches update
sub list/show command.

 1st patch is independent and updates man doc of btrfs-subvolume

 2nd-6th update the libbtrfsutil using new ioctls:
   - Relax the privileges of following functions if kernel supports new
     ioctls and @top/@id is zero (i.e. the given path/fd is used instead
     of arbitrary subvolume id).
     - util_subvolume_info()
     - subvolume iterator related ones (util_subvolume_iterator_next() etc.)

   - For subvolume iterator, if kernel supports new ioctls and @top is zero,
     non-subvolume directory can be specified as a start point. Also,
     subvolume which cannot be accessed (either because of permission
     error or not found (may happen if other volume is mounted in the
     path) will be skipped for non-privileged user.

   - Code path of root and non-privileged user is different. While root uses
     TREE_SEARCH ioctl as before, non-privileged user uses newly added
     ioctls. However, There is only one exception and when subvolume
     iterator is created from non-subvolume directory, code path of both is
     the same (and thus both use new ioctls).

 7th patch update the "sub list" to use libbtrfsutil (no behavior change yet)
   This is a copy of non-merged following patch originally written
   by Omar Sandoval:
     btrfs-progs: use libbtrfsutil for subvolume list [1]
   expect this commit keeps libbtrfs implementation which above commit
   tries to remove.

   (I suspect that the part of the reason that the original patch has not
   been merged is it removes libbtrfs and this commits modify this. but
   I'm completely fine with the original patch instead of this.)

 8th-15th patch update the behavior of "sub list"

 16th-17th patch update the behavior of "sub show"

 18th-20th patch are cli-test for "sub list" of new behavior.


* Future todo:
If this approach is ok, I'd like to update the output of "sub list" more like:
  - Remove obsolete field (i.e. top-level) from output

Any comments are welcome.
Thanks,
Tomohiro Misono

[1] https://www.spinics.net/lists/linux-btrfs/msg74917.html 

Misono Tomohiro (20):
  btrfs-progs: doc: Update man btrfs subvolume
  btrfs-progs: ioctl/libbtrfsutil: Add 3 definitions of new unprivileged
    ioctl
  btrfs-progs: libbtrfsutil: Factor out btrfs_util_subvolume_info_fd()
  btrfs-porgs: libbtrfsutil: Relax the privileges of
    util_subvolume_info()
  btrfs-progs: libbtrfsuitl: Factor out
    btrfs_util_subvolume_iterator_next()
  btrfs-progs: libbtrfsutil: Relax the privileges of subvolume iterator
  btrfs-progs: sub list: Use libbtrfsuitl for subvolume list
  btrfs-progs: sub list: factor out main part of btrfs_list_subvols
  btrfs-progs: sub list: Change the default behavior of "subvolume list"
    and allow non-privileged user to call it
  btrfs-progs: sub list: Add -A option to output path in absolute path
  btrfs-progs: sub list: Add -f option to follow mounted subvolumes
    below the path
  btrfs-progs: sub list: Add --nosort option to output incrementally
    without sort
  btrfs-progs: sub list: Update -a option and remove meaningless filter
  btrfs-progs: sub list: Update help message of -o option
  btrfs-progs: sub list: Update help message of -d option
  btrfs-progs: utils: Fallback to open without O_NOATIME flag in
    find_mount_root():
  btrfs-progs: sub show: Allow non-privileged user to call "subvolume
    show"
  btrfs-progs: test: Add helper function to check if test user exists
  btrfs-porgs: test: Add cli-test/009 to check subvolume list for both
    root and normal user
  btrfs-progs: test: Add cli-test/010 to check "subvolume list -f"
    option

 Documentation/btrfs-subvolume.asciidoc            |  108 +-
 cmds-subvolume.c                                  | 1372 ++++++++++++++++++++-
 ioctl.h                                           |   99 ++
 libbtrfsutil/btrfs.h                              |   97 ++
 libbtrfsutil/btrfsutil.h                          |   25 +-
 libbtrfsutil/errors.c                             |   10 +
 libbtrfsutil/subvolume.c                          |  494 +++++++-
 tests/cli-tests/009-subvolume-list/test.sh        |  134 ++
 tests/cli-tests/010-subvolume-list-follow/test.sh |   86 ++
 tests/common                                      |   10 +
 utils.c                                           |    3 +
 11 files changed, 2315 insertions(+), 123 deletions(-)
 create mode 100755 tests/cli-tests/009-subvolume-list/test.sh
 create mode 100755 tests/cli-tests/010-subvolume-list-follow/test.sh

-- 
2.14.4

[PATCH v2 02/20] btrfs-progs: ioctl/libbtrfsutil: Add 3 definitions of new unprivileged ioctl

From: Misono Tomohiro <hidden>
Date: 2018-06-18 08:38:12

Copy and add 3 definitions of new unprivileged ioctl
(BTRFS_IOC_GET_SUBVOL_INFO, BTRFS_IOC_GET_SUBVOL_ROOTREF and
BTRFS_IOC_INO_LOOKUP_USER) from kernel. They will be used to implement
the user version of "btrfs subvolume list/show" etc.

Signed-off-by: Misono Tomohiro <redacted>
---
 ioctl.h              | 99 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 libbtrfsutil/btrfs.h | 97 ++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 196 insertions(+)
diff --git a/ioctl.h b/ioctl.h
index 709e996f..cce55dbd 100644
--- a/ioctl.h
+++ b/ioctl.h
@@ -320,6 +320,22 @@ struct btrfs_ioctl_ino_lookup_args {
 };
 BUILD_ASSERT(sizeof(struct btrfs_ioctl_ino_lookup_args) == 4096);
 
+#define BTRFS_INO_LOOKUP_USER_PATH_MAX (4080 - BTRFS_VOL_NAME_MAX - 1)
+struct btrfs_ioctl_ino_lookup_user_args {
+	/* in, inode number containing the subvolume of 'subvolid' */
+	__u64 dirid;
+	/* in */
+	__u64 treeid;
+	/* out, name of the subvolume of 'treeid' */
+	char name[BTRFS_VOL_NAME_MAX + 1];
+	/*
+	 * out, constructed path from the directory with which the ioctl is
+	 * called to dirid
+	 */
+	char path[BTRFS_INO_LOOKUP_USER_PATH_MAX];
+};
+BUILD_ASSERT(sizeof(struct btrfs_ioctl_ino_lookup_user_args) == 4096);
+
 struct btrfs_ioctl_search_key {
 	/* which root are we searching.  0 is the tree of tree roots */
 	__u64 tree_id;
@@ -672,6 +688,83 @@ BUILD_ASSERT(sizeof(struct btrfs_ioctl_send_args_64) == 72);
 
 #define BTRFS_IOC_SEND_64_COMPAT_DEFINED 1
 
+/*
+ * Information about a fs tree root.
+ *
+ * All items are filled by the ioctl
+ */
+struct btrfs_ioctl_get_subvol_info_args {
+	/* Id of this subvolume */
+	__u64 treeid;
+
+	/* Name of this subvolume, used to get the real name at mount point */
+	char name[BTRFS_VOL_NAME_MAX + 1];
+
+	/*
+	 * Id of the subvolume which contains this subvolume.
+	 * Zero for top-level subvolume or a deleted subvolume.
+	 */
+	__u64 parent_id;
+
+	/*
+	 * Inode number of the directory which contains this subvolume.
+	 * Zero for top-level subvolume or a deleted subvolume
+	 */
+	__u64 dirid;
+
+	/* Latest transaction id of this subvolume */
+	__u64 generation;
+
+	/* Flags of this subvolume */
+	__u64 flags;
+
+	/* UUID of this subvolume */
+	__u8 uuid[BTRFS_UUID_SIZE];
+
+	/*
+	 * UUID of the subvolume of which this subvolume is a snapshot.
+	 * All zero for a non-snapshot subvolume.
+	 */
+	__u8 parent_uuid[BTRFS_UUID_SIZE];
+
+	/*
+	 * UUID of the subvolume from which this subvolume was received.
+	 * All zero for non-received subvolume.
+	 */
+	__u8 received_uuid[BTRFS_UUID_SIZE];
+
+	/* Transaction id indicating when change/create/send/receive happened */
+	__u64 ctransid;
+	__u64 otransid;
+	__u64 stransid;
+	__u64 rtransid;
+	/* Time corresponding to c/o/s/rtransid */
+	struct btrfs_ioctl_timespec ctime;
+	struct btrfs_ioctl_timespec otime;
+	struct btrfs_ioctl_timespec stime;
+	struct btrfs_ioctl_timespec rtime;
+
+	/* Must be zero */
+	__u64 reserved[8];
+};
+
+#define BTRFS_MAX_ROOTREF_BUFFER_NUM 255
+struct btrfs_ioctl_get_subvol_rootref_args {
+		/* in/out, minimum id of rootref's treeid to be searched */
+		__u64 min_treeid;
+
+		/* out */
+		struct {
+			__u64 treeid;
+			__u64 dirid;
+		} rootref[BTRFS_MAX_ROOTREF_BUFFER_NUM];
+
+		/* out, number of found items */
+		__u8 num_items;
+		__u8 align[7];
+};
+BUILD_ASSERT(sizeof(struct btrfs_ioctl_get_subvol_rootref_args) == 4096);
+
 /* Error codes as returned by the kernel */
 enum btrfs_err_code {
 	notused,
@@ -828,6 +921,12 @@ static inline char *btrfs_err_str(enum btrfs_err_code err_code)
                                   struct btrfs_ioctl_feature_flags[3])
 #define BTRFS_IOC_RM_DEV_V2	_IOW(BTRFS_IOCTL_MAGIC, 58, \
 				   struct btrfs_ioctl_vol_args_v2)
+#define BTRFS_IOC_GET_SUBVOL_INFO _IOR(BTRFS_IOCTL_MAGIC, 60, \
+				struct btrfs_ioctl_get_subvol_info_args)
+#define BTRFS_IOC_GET_SUBVOL_ROOTREF _IOWR(BTRFS_IOCTL_MAGIC, 61, \
+				struct btrfs_ioctl_get_subvol_rootref_args)
+#define BTRFS_IOC_INO_LOOKUP_USER _IOWR(BTRFS_IOCTL_MAGIC, 62, \
+				struct btrfs_ioctl_ino_lookup_user_args)
 #ifdef __cplusplus
 }
 #endif
diff --git a/libbtrfsutil/btrfs.h b/libbtrfsutil/btrfs.h
index c293f6bf..95740de2 100644
--- a/libbtrfsutil/btrfs.h
+++ b/libbtrfsutil/btrfs.h
@@ -421,6 +421,21 @@ struct btrfs_ioctl_ino_lookup_args {
 	char name[BTRFS_INO_LOOKUP_PATH_MAX];
 };
 
+#define BTRFS_INO_LOOKUP_USER_PATH_MAX (4080-BTRFS_VOL_NAME_MAX-1)
+struct btrfs_ioctl_ino_lookup_user_args {
+	/* in, inode number containing the subvolume of 'subvolid' */
+	__u64 dirid;
+	/* in */
+	__u64 treeid;
+	/* out, name of the subvolume of 'treeid' */
+	char name[BTRFS_VOL_NAME_MAX + 1];
+	/*
+	 * out, constructed path from the directory with which the ioctl is
+	 * called to dirid
+	 */
+	char path[BTRFS_INO_LOOKUP_USER_PATH_MAX];
+};
+
 /* Search criteria for the btrfs SEARCH ioctl family. */
 struct btrfs_ioctl_search_key {
 	/*
@@ -720,6 +735,82 @@ struct btrfs_ioctl_send_args {
 	__u64 reserved[4];		/* in */
 };
 
+/*
+ * Information about a fs tree root.
+ *
+ * All items are filled by the ioctl
+ */
+struct btrfs_ioctl_get_subvol_info_args {
+	/* Id of this subvolume */
+	__u64 treeid;
+
+	/* Name of this subvolume, used to get the real name at mount point */
+	char name[BTRFS_VOL_NAME_MAX + 1];
+
+	/*
+	 * Id of the subvolume which contains this subvolume.
+	 * Zero for top-level subvolume or a deleted subvolume.
+	 */
+	__u64 parent_id;
+
+	/*
+	 * Inode number of the directory which contains this subvolume.
+	 * Zero for top-level subvolume or a deleted subvolume
+	 */
+	__u64 dirid;
+
+	/* Latest transaction id of this subvolume */
+	__u64 generation;
+
+	/* Flags of this subvolume */
+	__u64 flags;
+
+	/* UUID of this subvolume */
+	__u8 uuid[BTRFS_UUID_SIZE];
+
+	/*
+	 * UUID of the subvolume of which this subvolume is a snapshot.
+	 * All zero for a non-snapshot subvolume.
+	 */
+	__u8 parent_uuid[BTRFS_UUID_SIZE];
+
+	/*
+	 * UUID of the subvolume from which this subvolume was received.
+	 * All zero for non-received subvolume.
+	 */
+	__u8 received_uuid[BTRFS_UUID_SIZE];
+
+	/* Transaction id indicating when change/create/send/receive happened */
+	__u64 ctransid;
+	__u64 otransid;
+	__u64 stransid;
+	__u64 rtransid;
+	/* Time corresponding to c/o/s/rtransid */
+	struct btrfs_ioctl_timespec ctime;
+	struct btrfs_ioctl_timespec otime;
+	struct btrfs_ioctl_timespec stime;
+	struct btrfs_ioctl_timespec rtime;
+
+	/* Must be zero */
+	__u64 reserved[8];
+};
+
+#define BTRFS_MAX_ROOTREF_BUFFER_NUM 255
+struct btrfs_ioctl_get_subvol_rootref_args {
+		/* in/out, minimum id of rootref's treeid to be searched */
+		__u64 min_treeid;
+
+		/* out */
+		struct {
+			__u64 treeid;
+			__u64 dirid;
+		} rootref[BTRFS_MAX_ROOTREF_BUFFER_NUM];
+
+		/* out, number of found items */
+		__u8 num_items;
+		__u8 align[7];
+};
+
 /* Error codes as returned by the kernel */
 enum btrfs_err_code {
 	BTRFS_ERROR_DEV_RAID1_MIN_NOT_MET = 1,
@@ -836,5 +927,11 @@ enum btrfs_err_code {
 				   struct btrfs_ioctl_feature_flags[3])
 #define BTRFS_IOC_RM_DEV_V2 _IOW(BTRFS_IOCTL_MAGIC, 58, \
 				   struct btrfs_ioctl_vol_args_v2)
+#define BTRFS_IOC_GET_SUBVOL_INFO _IOR(BTRFS_IOCTL_MAGIC, 60, \
+				struct btrfs_ioctl_get_subvol_info_args)
+#define BTRFS_IOC_GET_SUBVOL_ROOTREF _IOWR(BTRFS_IOCTL_MAGIC, 61, \
+				struct btrfs_ioctl_get_subvol_rootref_args)
+#define BTRFS_IOC_INO_LOOKUP_USER _IOWR(BTRFS_IOCTL_MAGIC, 62, \
+				struct btrfs_ioctl_ino_lookup_user_args)
 
 #endif /* _LINUX_BTRFS_H */
-- 
2.14.4

[PATCH v2 01/20] btrfs-progs: doc: Update man btrfs subvolume

From: Misono Tomohiro <hidden>
Date: 2018-06-18 08:38:13

Some information is obsolete and updated as follows:
 - Add missing explanations of some options
 - Remove outdated explanation of "subvolrootid" mount option
 - Reorder/group options of "sub list" to corresponds help message
 - Add explanation about different meaning of parent in "parent ID/UUID"
 - Fix indent/misspelling
 - Add missing comma

Signed-off-by: Misono Tomohiro <redacted>
---
 Documentation/btrfs-subvolume.asciidoc | 77 ++++++++++++++++++++++++----------
 1 file changed, 55 insertions(+), 22 deletions(-)
diff --git a/Documentation/btrfs-subvolume.asciidoc b/Documentation/btrfs-subvolume.asciidoc
index a8c4af4b..f3eb4e26 100644
--- a/Documentation/btrfs-subvolume.asciidoc
+++ b/Documentation/btrfs-subvolume.asciidoc
@@ -77,13 +77,16 @@ safely stored on the device.
 `Options`
 +
 -c|--commit-after::::
-wait for transaction commit at the end of the operation
+wait for transaction commit at the end of the operation.
 +
 -C|--commit-each::::
-wait for transaction commit after deleting each subvolume
+wait for transaction commit after deleting each subvolume.
++
+-v|--verbose::::
+verbose output of operations.
 
 *find-new* <subvolume> <last_gen>::
-List the recently modified files in a subvolume, after <last_gen> ID.
+List the recently modified files in a subvolume, after <last_gen> generation.
 
 *get-default* <path>::
 Get the default subvolume of the filesystem <path>.
@@ -93,40 +96,54 @@ The output format is similar to *subvolume list* command.
 *list* [options] [-G [\+|-]<value>] [-C [+|-]<value>] [--sort=rootid,gen,ogen,path] <path>::
 List the subvolumes present in the filesystem <path>.
 +
-For every subvolume the following information is shown by default. +
-ID <ID> top level <ID> path <path> +
-where path is the relative path of the subvolume to the top level subvolume.
+For every subvolume the following information is shown by default:
++
+ID <ID> gen <generation> top level <ID> path <path>
++
+where ID is subvolume's id, gen is an internal counter which is updated
+every transaction, top level is the same as parent subvolume's id, and
+path is the relative path of the subvolume to the top level subvolume.
 The subvolume's ID may be used by the subvolume set-default command,
 or at mount time via the subvolid= option.
-If `-p` is given, then parent <ID> is added to the output between ID
-and top level. The parent's ID may be used at mount time via the
-`subvolrootid=` option.
 +
 `Options`
 +
--p::::
-print parent ID.
+Path filtering;;
+-o::::
+print only subvolumes below specified <path>.
 -a::::
 print all the subvolumes in the filesystem and distinguish between
 absolute and relative path with respect to the given <path>.
+
+Field selection;;
+-p::::
+print the parent ID
+('parent' here means the subvolume which contains this subvolume).
 -c::::
 print the ogeneration of the subvolume, aliases: ogen or origin generation.
 -g::::
-print the generation of the subvolume.
--o::::
-print only subvolumes below specified <path>.
+print the generation of the subvolume (default).
 -u::::
 print the UUID of the subvolume.
 -q::::
-print the parent uuid of subvolumes (and snapshots).
+print the parent UUID of the subvolume
+('parent' here means subvolume of which this subvolume is a snapshot).
 -R::::
-print the UUID of the sent subvolume, where the subvolume is the result of a receive operation
--t::::
-print the result as a table.
+print the UUID of the sent subvolume, where the subvolume is the result of a receive operation.
+
+Type filtering;;
 -s::::
 only snapshot subvolumes in the filesystem will be listed.
 -r::::
 only readonly subvolumes in the filesystem will be listed.
+-d::::
+list deleted subvolumes that are not yet cleaned.
+
+Other;;
+-t::::
+print the result as a table.
+
+Sorting;;
 -G [+|-]<value>::::
 list subvolumes in the filesystem that its generation is
 >=, \<= or = value. \'\+' means >= value, \'-' means \<= value, If there is
@@ -144,9 +161,9 @@ for --sort you can combine some items together by \',', just like
 
 *set-default* [<subvolume>|<id> <path>]::
 Set the default subvolume for the (mounted) filesystem.
-
++
 Set the default subvolume for the (mounted) filesystem at <path>. This will hide
-the top-level subvolume (ie. the one mounted with 'subvol=/' or 'subvolid=5').
+the top-level subvolume (i.e. the one mounted with 'subvol=/' or 'subvolid=5').
 Takes action on next mount.
 +
 There are two ways how to specify the subvolume, by <id> or by the <subvolume>
@@ -154,10 +171,22 @@ path.
 The id can be obtained from *btrfs subvolume list*, *btrfs subvolume show* or
 *btrfs inspect-internal rootid*.
 
-*show* <path>::
+*show* [options] <path>|<mnt>::
 Show information of a given subvolume in the <path>.
++
+`Options`
++
+-r|--rootid::::
+rootid of the subvolume.
+-u|--uuid:::
+UUID of the subvolume.
+
++
+If no option is specified, subvolume information of <path> is shown,
+otherwise the subvolume information of rootid or UUID in the filesystem
+is shown.
 
-*snapshot* [-r] <source> <dest>|[<dest>/]<name>::
+*snapshot* [-r|-i <qgroupid>] <source> <dest>|[<dest>/]<name>::
 Create a snapshot of the subvolume <source> with the
 name <name> in the <dest> directory.
 +
@@ -168,6 +197,10 @@ If <source> is not a subvolume, btrfs returns an error.
 +
 -r::::
 Make the new snapshot read only.
++
+-i <qgroupid>::::
+Add the newly created subvolume to a qgroup. This option can be given multiple
+times.
 
 *sync* <path> [subvolid...]::
 Wait until given subvolume(s) are completely removed from the filesystem after
-- 
2.14.4

[PATCH v2 03/20] btrfs-progs: libbtrfsutil: Factor out btrfs_util_subvolume_info_fd()

From: Misono Tomohiro <hidden>
Date: 2018-06-18 08:38:13

Factor out main logic of btrfs_util_subvolume_info_fd().
This is a preparation work to relax the root privilege of this function.

No functional change happens.

Signed-off-by: Misono Tomohiro <redacted>
---
 libbtrfsutil/subvolume.c | 45 ++++++++++++++++++++++++++-------------------
 1 file changed, 26 insertions(+), 19 deletions(-)
diff --git a/libbtrfsutil/subvolume.c b/libbtrfsutil/subvolume.c
index 867b3e10..0d7ef5bf 100644
--- a/libbtrfsutil/subvolume.c
+++ b/libbtrfsutil/subvolume.c
@@ -295,8 +295,8 @@ PUBLIC enum btrfs_util_error btrfs_util_subvolume_info(const char *path,
 	return err;
 }
 
-PUBLIC enum btrfs_util_error btrfs_util_subvolume_info_fd(int fd, uint64_t id,
-							  struct btrfs_util_subvolume_info *subvol)
+static enum btrfs_util_error get_subvolume_info_root(int fd, uint64_t id,
+						     struct btrfs_util_subvolume_info *subvol)
 {
 	struct btrfs_ioctl_search_args search = {
 		.key = {
@@ -310,27 +310,10 @@ PUBLIC enum btrfs_util_error btrfs_util_subvolume_info_fd(int fd, uint64_t id,
 			.nr_items = 0,
 		},
 	};
-	enum btrfs_util_error err;
 	size_t items_pos = 0, buf_off = 0;
 	bool need_root_item = true, need_root_backref = true;
 	int ret;
 
-	if (id == 0) {
-		err = btrfs_util_is_subvolume_fd(fd);
-		if (err)
-			return err;
-
-		err = btrfs_util_subvolume_id_fd(fd, &id);
-		if (err)
-			return err;
-	}
-
-	if ((id < BTRFS_FIRST_FREE_OBJECTID && id != BTRFS_FS_TREE_OBJECTID) ||
-	    id > BTRFS_LAST_FREE_OBJECTID) {
-		errno = ENOENT;
-		return BTRFS_UTIL_ERROR_SUBVOLUME_NOT_FOUND;
-	}
-
 	search.key.min_objectid = search.key.max_objectid = id;
 
 	if (subvol) {
@@ -400,6 +383,30 @@ PUBLIC enum btrfs_util_error btrfs_util_subvolume_info_fd(int fd, uint64_t id,
 	return BTRFS_UTIL_OK;
 }
 
+PUBLIC enum btrfs_util_error btrfs_util_subvolume_info_fd(int fd, uint64_t id,
+							  struct btrfs_util_subvolume_info *subvol)
+{
+	enum btrfs_util_error err;
+
+	if (id == 0) {
+		err = btrfs_util_is_subvolume_fd(fd);
+		if (err)
+			return err;
+
+		err = btrfs_util_subvolume_id_fd(fd, &id);
+		if (err)
+			return err;
+	}
+
+	if ((id < BTRFS_FIRST_FREE_OBJECTID && id != BTRFS_FS_TREE_OBJECTID) ||
+	    id > BTRFS_LAST_FREE_OBJECTID) {
+		errno = ENOENT;
+		return BTRFS_UTIL_ERROR_SUBVOLUME_NOT_FOUND;
+	}
+
+	return get_subvolume_info_root(fd, id, subvol);
+}
+
 PUBLIC enum btrfs_util_error btrfs_util_get_subvolume_read_only_fd(int fd,
 								   bool *read_only_ret)
 {
-- 
2.14.4

[PATCH v2 04/20] btrfs-porgs: libbtrfsutil: Relax the privileges of util_subvolume_info()

From: Misono Tomohiro <hidden>
Date: 2018-06-18 08:38:13

This commit relaxes the privileges of util_subvolume_info() if kernel
supports new ioctl (BTRFS_IOC_GET_SUBVOL_INFO) and @id is zero
(i.e. when getting the information of the given path/fd).
For older kernel (< 4.18), the behavior is the same.

Signed-off-by: Misono Tomohiro <redacted>
---
 libbtrfsutil/btrfsutil.h |  7 +++++-
 libbtrfsutil/errors.c    |  4 ++++
 libbtrfsutil/subvolume.c | 58 ++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 68 insertions(+), 1 deletion(-)
diff --git a/libbtrfsutil/btrfsutil.h b/libbtrfsutil/btrfsutil.h
index 6d655f49..3f21d163 100644
--- a/libbtrfsutil/btrfsutil.h
+++ b/libbtrfsutil/btrfsutil.h
@@ -63,6 +63,8 @@ enum btrfs_util_error {
 	BTRFS_UTIL_ERROR_SYNC_FAILED,
 	BTRFS_UTIL_ERROR_START_SYNC_FAILED,
 	BTRFS_UTIL_ERROR_WAIT_SYNC_FAILED,
+	BTRFS_UTIL_ERROR_INVALID_ARGUMENT_FOR_USER,
+	BTRFS_UTIL_ERROR_GET_SUBVOL_INFO_FAILED,
 };
 
 /**
@@ -266,7 +268,10 @@ struct btrfs_util_subvolume_info {
  * to check whether the subvolume exists; %BTRFS_UTIL_ERROR_SUBVOLUME_NOT_FOUND
  * will be returned if it does not.
  *
- * This requires appropriate privilege (CAP_SYS_ADMIN).
+ * This requires appropriate privilege (CAP_SYS_ADMIN) for kernel < 4.18.
+ * From kernel >= 4.18 which supports BTRFS_IOC_GET_SUGBVOL_INFO,
+ * non-privileged user with appropriate permission for @path can use this too
+ * (in that case @id must be zero).
  *
  * Return: %BTRFS_UTIL_OK on success, non-zero error code on failure.
  */
diff --git a/libbtrfsutil/errors.c b/libbtrfsutil/errors.c
index 634edc65..f196fa71 100644
--- a/libbtrfsutil/errors.c
+++ b/libbtrfsutil/errors.c
@@ -45,6 +45,10 @@ static const char * const error_messages[] = {
 	[BTRFS_UTIL_ERROR_SYNC_FAILED] = "Could not sync filesystem",
 	[BTRFS_UTIL_ERROR_START_SYNC_FAILED] = "Could not start filesystem sync",
 	[BTRFS_UTIL_ERROR_WAIT_SYNC_FAILED] = "Could not wait for filesystem sync",
+	[BTRFS_UTIL_ERROR_INVALID_ARGUMENT_FOR_USER] =
+		"Non-root user cannot specify subvolume id",
+	[BTRFS_UTIL_ERROR_GET_SUBVOL_INFO_FAILED] =
+	"Could not get subvolume information by BTRFS_IOC_GET_SUBVOL_INFO",
 };
 
 PUBLIC const char *btrfs_util_strerror(enum btrfs_util_error err)
diff --git a/libbtrfsutil/subvolume.c b/libbtrfsutil/subvolume.c
index 0d7ef5bf..e94c7079 100644
--- a/libbtrfsutil/subvolume.c
+++ b/libbtrfsutil/subvolume.c
@@ -31,6 +31,14 @@
 
 #include "btrfsutil_internal.h"
 
+static bool is_root(void)
+{
+	uid_t uid;
+
+	uid = geteuid();
+	return (uid == 0);
+}
+
 /*
  * This intentionally duplicates btrfs_util_is_subvolume_fd() instead of opening
  * a file descriptor and calling it, because fstat() and fstatfs() don't accept
@@ -383,11 +391,61 @@ static enum btrfs_util_error get_subvolume_info_root(int fd, uint64_t id,
 	return BTRFS_UTIL_OK;
 }
 
+static enum btrfs_util_error get_subvolume_info_user(int fd,
+						     struct btrfs_util_subvolume_info *subvol)
+{
+	struct btrfs_ioctl_get_subvol_info_args info;
+	int ret;
+
+	ret = ioctl(fd, BTRFS_IOC_GET_SUBVOL_INFO, &info);
+	if (ret < 0)
+		return BTRFS_UTIL_ERROR_GET_SUBVOL_INFO_FAILED;
+
+	subvol->id = info.treeid;
+	subvol->parent_id = info.parent_id;
+	subvol->dir_id = info.dirid;
+	subvol->flags = info.flags;
+	subvol->generation = info.generation;
+
+	memcpy(subvol->uuid, info.uuid, sizeof(subvol->uuid));
+	memcpy(subvol->parent_uuid, info.parent_uuid,
+			sizeof(subvol->parent_uuid));
+	memcpy(subvol->received_uuid, info.received_uuid,
+			sizeof(subvol->received_uuid));
+
+	subvol->ctransid = info.ctransid;
+	subvol->otransid = info.otransid;
+	subvol->stransid = info.stransid;
+	subvol->rtransid = info.rtransid;
+
+	subvol->ctime.tv_sec  = info.ctime.sec;
+	subvol->ctime.tv_nsec = info.ctime.nsec;
+	subvol->otime.tv_sec  = info.otime.sec;
+	subvol->otime.tv_nsec = info.otime.nsec;
+	subvol->stime.tv_sec  = info.stime.sec;
+	subvol->stime.tv_nsec = info.stime.nsec;
+	subvol->rtime.tv_sec  = info.rtime.sec;
+	subvol->rtime.tv_nsec = info.rtime.nsec;
+
+	return BTRFS_UTIL_OK;
+}
+
 PUBLIC enum btrfs_util_error btrfs_util_subvolume_info_fd(int fd, uint64_t id,
 							  struct btrfs_util_subvolume_info *subvol)
 {
 	enum btrfs_util_error err;
 
+	if (!is_root()) {
+		if (id != 0)
+			return BTRFS_UTIL_ERROR_INVALID_ARGUMENT_FOR_USER;
+
+		err = btrfs_util_is_subvolume_fd(fd);
+		if (err)
+			return err;
+
+		return get_subvolume_info_user(fd, subvol);
+	}
+
 	if (id == 0) {
 		err = btrfs_util_is_subvolume_fd(fd);
 		if (err)
-- 
2.14.4

[PATCH v2 05/20] btrfs-progs: libbtrfsuitl: Factor out btrfs_util_subvolume_iterator_next()

From: Misono Tomohiro <hidden>
Date: 2018-06-18 08:38:13

Factor out the main logic of btrfs_util_subvolume_iterator_next().
This is a preparation work to relax the root privilege of this function.

No functional change happens.

Signed-off-by: Misono Tomohiro <redacted>
---
 libbtrfsutil/subvolume.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)
diff --git a/libbtrfsutil/subvolume.c b/libbtrfsutil/subvolume.c
index e94c7079..73471d4f 100644
--- a/libbtrfsutil/subvolume.c
+++ b/libbtrfsutil/subvolume.c
@@ -1255,9 +1255,9 @@ static enum btrfs_util_error build_subvol_path(struct btrfs_util_subvolume_itera
 	return BTRFS_UTIL_OK;
 }
 
-PUBLIC enum btrfs_util_error btrfs_util_subvolume_iterator_next(struct btrfs_util_subvolume_iterator *iter,
-								char **path_ret,
-								uint64_t *id_ret)
+static enum btrfs_util_error subvolume_iterator_next_root(struct btrfs_util_subvolume_iterator *iter,
+							  char **path_ret,
+							  uint64_t *id_ret)
 {
 	struct search_stack_entry *top;
 	const struct btrfs_ioctl_search_header *header;
@@ -1331,6 +1331,13 @@ out:
 	return BTRFS_UTIL_OK;
 }
 
+PUBLIC enum btrfs_util_error btrfs_util_subvolume_iterator_next(struct btrfs_util_subvolume_iterator *iter,
+								char **path_ret,
+								uint64_t *id_ret)
+{
+	return subvolume_iterator_next_root(iter, path_ret, id_ret);
+}
+
 PUBLIC enum btrfs_util_error btrfs_util_subvolume_iterator_next_info(struct btrfs_util_subvolume_iterator *iter,
 								     char **path_ret,
 								     struct btrfs_util_subvolume_info *subvol)
-- 
2.14.4

[PATCH v2 10/20] btrfs-progs: sub list: Add -A option to output path in absolute path

From: Misono Tomohiro <hidden>
Date: 2018-06-18 08:38:13

By default, the printed path is relative to the specified path.
Let's add option to print absolute path with -A option.

[Example]
 $ mkfs.btrfs -f $DEV
 $ mount $DEV /mnt

 $ btrfs subvolume create /mnt/AAA

 $ btrfs subvolume list /mnt
 ID 256 gen 6 top level 5 path AAA

 $ btrfs subvolume list -A /mnt
 ID 256 gen 6 top level 5 path /mnt/AAA

Signed-off-by: Misono Tomohiro <redacted>
---
 Documentation/btrfs-subvolume.asciidoc |  2 +
 cmds-subvolume.c                       | 73 +++++++++++++++++++++++++++++-----
 2 files changed, 66 insertions(+), 9 deletions(-)
diff --git a/Documentation/btrfs-subvolume.asciidoc b/Documentation/btrfs-subvolume.asciidoc
index 99fff977..fec4b769 100644
--- a/Documentation/btrfs-subvolume.asciidoc
+++ b/Documentation/btrfs-subvolume.asciidoc
@@ -148,6 +148,8 @@ list deleted subvolumes that are not yet cleaned.
 Other;;
 -t::::
 print the result as a table.
+-A::::
+print path in absolute path.
 
 Sorting;;
 -G [+|-]<value>::::
diff --git a/cmds-subvolume.c b/cmds-subvolume.c
index 23596c17..ea341d50 100644
--- a/cmds-subvolume.c
+++ b/cmds-subvolume.c
@@ -1156,7 +1156,8 @@ static void get_subvols_info(struct subvol_list **subvols,
 			     struct btrfs_list_filter_set_v2 *filter_set,
 			     int fd,
 			     int tree_id,
-			     size_t *capacity)
+			     size_t *capacity,
+			     const char *prefix)
 {
 	struct btrfs_util_subvolume_iterator *iter;
 	enum btrfs_util_error err;
@@ -1206,7 +1207,10 @@ static void get_subvols_info(struct subvol_list **subvols,
 			goto out;
 		}
 
-		subvol.path = strdup(".");
+		if (prefix)
+			subvol.path = strdup(prefix);
+		else
+			subvol.path = strdup(".");
 		if (!filters_match(&subvol, filter_set)) {
 			free(subvol.path);
 		} else {
@@ -1231,6 +1235,29 @@ skip:
 			goto out;
 		}
 
+		if (prefix) {
+			char *temp = subvol.path;
+
+			subvol.path = malloc(strlen(prefix) +
+					     strlen(subvol.path) + 2);
+			if (!subvol.path) {
+				error("out of memory");
+				subvol.path = temp;
+				ret = -1;
+				goto out;
+			}
+
+			strcpy(subvol.path, prefix);
+			if (strlen(prefix) == 1) {
+				strcpy(subvol.path + 1, temp);
+			} else {
+				subvol.path[strlen(prefix)] = '/';
+				strcpy(subvol.path + strlen(prefix) + 1, temp);
+			}
+
+			free(temp);
+		}
+
 		if (!filters_match(&subvol, filter_set)) {
 			free(subvol.path);
 		} else {
@@ -1252,6 +1279,7 @@ out:
 
 static struct subvol_list *btrfs_list_subvols(int fd,
 					      int is_list_all,
+					      int absolute_path,
 					      const char *path,
 					      struct btrfs_list_filter_set_v2 *filter_set)
 {
@@ -1265,11 +1293,24 @@ static struct subvol_list *btrfs_list_subvols(int fd,
 	}
 	subvols->num = 0;
 
-	if (is_list_all)
+	if (is_list_all) {
 		get_subvols_info(&subvols, filter_set, fd,
-				BTRFS_FS_TREE_OBJECTID, &capacity);
-	else
-		get_subvols_info(&subvols, filter_set, fd, 0, &capacity);
+				BTRFS_FS_TREE_OBJECTID, &capacity, NULL);
+	} else {
+		char *fullpath;
+
+		fullpath = realpath(path, NULL);
+		if (!fullpath) {
+			error("cannot find real path for '%s': %m", path);
+			free_subvol_list(subvols);
+			return NULL;
+		}
+
+		get_subvols_info(&subvols, filter_set, fd, 0, &capacity,
+				(absolute_path ? fullpath : NULL));
+
+		free(fullpath);
+	}
 
 	return subvols;
 }
@@ -1279,6 +1320,7 @@ static int btrfs_list_subvols_print_v2(int fd,
 				    struct btrfs_list_comparer_set_v2 *comp_set,
 				    enum btrfs_list_layout layout,
 				    int is_list_all,
+				    int absolute_path,
 				    const char *path,
 				    const char *raw_prefix)
 {
@@ -1287,7 +1329,8 @@ static int btrfs_list_subvols_print_v2(int fd,
 	if (filter_set->only_deleted)
 		subvols = btrfs_list_deleted_subvols(fd, filter_set);
 	else
-		subvols = btrfs_list_subvols(fd, is_list_all, path, filter_set);
+		subvols = btrfs_list_subvols(fd, is_list_all, absolute_path,
+					     path, filter_set);
 	if (!subvols)
 		return -1;
 
@@ -1427,6 +1470,8 @@ static const char * const cmd_subvol_list_usage[] = {
 	"",
 	"Other:",
 	"-t           print the result as a table",
+	"-A           print path in absolute path",
+	"             (default is a relative to the specified path)",
 	"",
 	"Sorting:",
 	"-G [+|-]value",
@@ -1453,6 +1498,7 @@ static int cmd_subvol_list(int argc, char **argv)
 	char *subvol;
 	int is_list_all = 0;
 	int is_only_in_path = 0;
+	int absolute_path = 0;
 	DIR *dirstream = NULL;
 	enum btrfs_list_layout layout = BTRFS_LIST_LAYOUT_DEFAULT;
 
@@ -1467,11 +1513,14 @@ static int cmd_subvol_list(int argc, char **argv)
 		};
 
 		c = getopt_long(argc, argv,
-				    "acdgopqsurRG:C:t", long_options, NULL);
+				    "acdgopqsurARG:C:t", long_options, NULL);
 		if (c < 0)
 			break;
 
 		switch(c) {
+		case 'A':
+			absolute_path = 1;
+			break;
 		case 'p':
 			btrfs_list_setup_print_column_v2(BTRFS_LIST_PARENT);
 			break;
@@ -1561,6 +1610,12 @@ static int cmd_subvol_list(int argc, char **argv)
 		goto out;
 	}
 
+	if (is_list_all && absolute_path) {
+		ret = -1;
+		error("cannot use -a with -A option");
+		goto out;
+	}
+
 	subvol = argv[optind];
 	fd = btrfs_open_dir(subvol, &dirstream, 1);
 	if (fd < 0) {
@@ -1593,7 +1648,7 @@ static int cmd_subvol_list(int argc, char **argv)
 	btrfs_list_setup_print_column_v2(BTRFS_LIST_PATH);
 
 	ret = btrfs_list_subvols_print_v2(fd, filter_set, comparer_set,
-			layout, is_list_all, subvol, NULL);
+			layout, is_list_all, absolute_path, subvol, NULL);
 
 out:
 	close_file_or_dir(fd, dirstream);
-- 
2.14.4

[PATCH v2 12/20] btrfs-progs: sub list: Add --nosort option to output incrementally without sort

From: Misono Tomohiro <hidden>
Date: 2018-06-18 08:38:16

Currently, "subvolume list" loads all the subvolume information into
memory first in order to sort them. This may cause a performance problem
if there are a lot of subvolumes.

This commit adds --nosort option to output subvolume information
incrementally without sort to avoid consuming memory.

Signed-off-by: Misono Tomohiro <redacted>
---
 Documentation/btrfs-subvolume.asciidoc |   5 ++
 cmds-subvolume.c                       | 140 ++++++++++++++++++++++-----------
 2 files changed, 100 insertions(+), 45 deletions(-)
diff --git a/Documentation/btrfs-subvolume.asciidoc b/Documentation/btrfs-subvolume.asciidoc
index b2461398..cd91385a 100644
--- a/Documentation/btrfs-subvolume.asciidoc
+++ b/Documentation/btrfs-subvolume.asciidoc
@@ -170,6 +170,11 @@ you can add \'\+' or \'-' in front of each items, \'+' means ascending,
 +
 for --sort you can combine some items together by \',', just like
 --sort=+ogen,-gen,path,rootid.
++
+--nosort::::
+Output the results incrementally without sort. This avoids loading all
+subvolume information to memory and can be useful when there is a lot
+of subvolumes.
 
 *set-default* [<subvolume>|<id> <path>]::
 Set the default subvolume for the (mounted) filesystem.
diff --git a/cmds-subvolume.c b/cmds-subvolume.c
index 4ebe0377..802f5c5e 100644
--- a/cmds-subvolume.c
+++ b/cmds-subvolume.c
@@ -1036,6 +1036,23 @@ static void print_all_subvol_info_tab_head(void)
 	}
 }
 
+static void print_one_subvol_info(struct listed_subvol *subvol,
+				  enum btrfs_list_layout layout,
+				  const char *raw_prefix)
+{
+	switch (layout) {
+	case BTRFS_LIST_LAYOUT_DEFAULT:
+		print_one_subvol_info_default(subvol);
+		break;
+	case BTRFS_LIST_LAYOUT_TABLE:
+		print_one_subvol_info_table(subvol);
+		break;
+	case BTRFS_LIST_LAYOUT_RAW:
+		print_one_subvol_info_raw(subvol, raw_prefix);
+		break;
+	}
+}
+
 static void print_all_subvol_info(struct subvol_list *subvols,
 				  enum btrfs_list_layout layout,
 				  const char *raw_prefix)
@@ -1048,17 +1065,7 @@ static void print_all_subvol_info(struct subvol_list *subvols,
 	for (i = 0; i < subvols->num; i++) {
 		struct listed_subvol *subvol = &subvols->subvols[i];
 
-		switch (layout) {
-		case BTRFS_LIST_LAYOUT_DEFAULT:
-			print_one_subvol_info_default(subvol);
-			break;
-		case BTRFS_LIST_LAYOUT_TABLE:
-			print_one_subvol_info_table(subvol);
-			break;
-		case BTRFS_LIST_LAYOUT_RAW:
-			print_one_subvol_info_raw(subvol, raw_prefix);
-			break;
-		}
+		print_one_subvol_info(subvol, layout, raw_prefix);
 	}
 }
 
@@ -1159,7 +1166,9 @@ static void get_subvols_info(struct subvol_list **subvols,
 			     int tree_id,
 			     size_t *capacity,
 			     const char *prefix,
-			     int show_top)
+			     int show_top,
+			     enum btrfs_list_layout layout,
+			     const char *raw_prefix)
 {
 	struct btrfs_util_subvolume_iterator *iter;
 	enum btrfs_util_error err;
@@ -1216,9 +1225,14 @@ static void get_subvols_info(struct subvol_list **subvols,
 		if (!filters_match(&subvol, filter_set)) {
 			free(subvol.path);
 		} else {
-			ret = add_subvol(subvols, &subvol, capacity);
-			if (ret)
-				goto out;
+			if (*subvols == NULL) {
+				print_one_subvol_info(&subvol,
+						      layout, raw_prefix);
+			} else {
+				ret = add_subvol(subvols, &subvol, capacity);
+				if (ret)
+					goto out;
+			}
 		}
 	}
 
@@ -1263,9 +1277,14 @@ skip:
 		if (!filters_match(&subvol, filter_set)) {
 			free(subvol.path);
 		} else {
-			ret = add_subvol(subvols, &subvol, capacity);
-			if (ret)
-				goto out;
+			if (*subvols == NULL) {
+				print_one_subvol_info(&subvol,
+						      layout, raw_prefix);
+			} else {
+				ret = add_subvol(subvols, &subvol, capacity);
+				if (ret)
+					goto out;
+			}
 		}
 	}
 
@@ -1275,7 +1294,7 @@ out:
 		btrfs_util_destroy_subvolume_iterator(iter);
 	if (ret) {
 		free_subvol_list(*subvols);
-		*subvols = NULL;
+		*subvols = ERR_PTR(ret);
 	}
 }
 
@@ -1283,23 +1302,31 @@ static struct subvol_list *btrfs_list_subvols(int fd,
 					      int is_list_all,
 					      int absolute_path,
 					      int follow_mount,
+					      int no_sort,
+					      enum btrfs_list_layout layout,
+					      const char *raw_prefix,
 					      const char *path,
 					      struct btrfs_list_filter_set_v2 *filter_set)
 {
-	struct subvol_list *subvols;
+	struct subvol_list *subvols = NULL;
 	size_t capacity = 0;
 
-	subvols = malloc(sizeof(*subvols));
-	if (!subvols) {
-		error("out of memory");
-		return NULL;
+	if (!no_sort) {
+		subvols = malloc(sizeof(*subvols));
+		if (!subvols) {
+			error("out of memory");
+			return ERR_PTR(-1);
+		}
+		subvols->num = 0;
 	}
-	subvols->num = 0;
+
+	if (no_sort && layout == BTRFS_LIST_LAYOUT_TABLE)
+		print_all_subvol_info_tab_head();
 
 	if (is_list_all) {
 		get_subvols_info(&subvols, filter_set, fd,
 				BTRFS_FS_TREE_OBJECTID, &capacity, NULL,
-				false);
+				false, layout, raw_prefix);
 	} else {
 		char *fullpath;
 
@@ -1307,16 +1334,17 @@ static struct subvol_list *btrfs_list_subvols(int fd,
 		if (!fullpath) {
 			error("cannot find real path for '%s': %m", path);
 			free_subvol_list(subvols);
-			return NULL;
+			return ERR_PTR(-1);
 		}
 
 		get_subvols_info(&subvols, filter_set, fd, 0, &capacity,
-				(absolute_path ? fullpath : NULL), false);
+				(absolute_path ? fullpath : NULL), false,
+				layout, raw_prefix);
 
-		if (subvols == NULL) {
-			free(fullpath);
-			return NULL;
-		}
+			if (IS_ERR(subvols)) {
+				free(fullpath);
+				return ERR_PTR(-1);
+			}
 
 		/* Follow mounted subvolumes below @path */
 		if (follow_mount) {
@@ -1334,7 +1362,7 @@ static struct subvol_list *btrfs_list_subvols(int fd,
 				error("failed to get fsid: %m");
 				free(fullpath);
 				free_subvol_list(subvols);
-				return NULL;
+				return ERR_PTR(-1);
 			}
 
 			f = setmntent("/proc/self/mounts", "r");
@@ -1342,7 +1370,7 @@ static struct subvol_list *btrfs_list_subvols(int fd,
 				error("failed to read mount entry: %m");
 				free(fullpath);
 				free_subvol_list(subvols);
-				return NULL;
+				return ERR_PTR(-1);
 			}
 
 			/* Iterate for each mount entry */
@@ -1369,7 +1397,7 @@ static struct subvol_list *btrfs_list_subvols(int fd,
 					error("failed to get fsid: %m");
 					free(fullpath);
 					free_subvol_list(subvols);
-					return NULL;
+					return ERR_PTR(-1);
 				}
 				if (uuid_compare(fsid, fsid2))
 					continue;
@@ -1381,7 +1409,7 @@ static struct subvol_list *btrfs_list_subvols(int fd,
 							mnt->mnt_dir);
 					free(fullpath);
 					free_subvol_list(subvols);
-					return NULL;
+					return ERR_PTR(-1);
 				}
 				get_subvols_info(&subvols, filter_set,
 						fd2, 0, &capacity,
@@ -1389,11 +1417,11 @@ static struct subvol_list *btrfs_list_subvols(int fd,
 							(strlen(fullpath) == 1 ?
 							 mnt->mnt_dir + 1 :
 							 mnt->mnt_dir + strlen(fullpath) + 1)),
-						true);
+						true, layout, raw_prefix);
 				close_file_or_dir(fd2, dirstream);
-				if (subvols == NULL) {
+				if (IS_ERR(subvols)) {
 					free(fullpath);
-					return NULL;
+					return ERR_PTR(-1);
 				}
 			}
 		}
@@ -1410,6 +1438,7 @@ static int btrfs_list_subvols_print_v2(int fd,
 				    int is_list_all,
 				    int absolute_path,
 				    int follow_mount,
+				    int no_sort,
 				    const char *path,
 				    const char *raw_prefix)
 {
@@ -1419,15 +1448,20 @@ static int btrfs_list_subvols_print_v2(int fd,
 		subvols = btrfs_list_deleted_subvols(fd, filter_set);
 	else
 		subvols = btrfs_list_subvols(fd, is_list_all, absolute_path,
-					     follow_mount, path, filter_set);
-	if (!subvols)
+					     follow_mount, no_sort,
+					     layout, raw_prefix,
+					     path, filter_set);
+
+	if (IS_ERR(subvols))
 		return -1;
 
-	sort_subvols(comp_set, subvols);
+	if (!no_sort) {
+		sort_subvols(comp_set, subvols);
 
-	print_all_subvol_info(subvols, layout, raw_prefix);
+		print_all_subvol_info(subvols, layout, raw_prefix);
 
-	free_subvol_list(subvols);
+		free_subvol_list(subvols);
+	}
 
 	return 0;
 }
@@ -1575,6 +1609,9 @@ static const char * const cmd_subvol_list_usage[] = {
 	"             list the subvolume in order of gen, ogen, rootid or path",
 	"             you also can add '+' or '-' in front of each items.",
 	"             (+:ascending, -:descending, ascending default)",
+	"--nosort     Output the results incrementally without sort.",
+	"             This avoids loading all subvolume information to memory",
+	"             and can be useful when there is a lot of subvolumes",
 	NULL,
 };
 
@@ -1589,6 +1626,8 @@ static int cmd_subvol_list(int argc, char **argv)
 	char *subvol;
 	int is_list_all = 0;
 	int follow_mount = 0;
+	int sort = 0;
+	int no_sort = 0;
 	int is_only_in_path = 0;
 	int absolute_path = 0;
 	DIR *dirstream = NULL;
@@ -1601,6 +1640,7 @@ static int cmd_subvol_list(int argc, char **argv)
 		int c;
 		static const struct option long_options[] = {
 			{"sort", required_argument, NULL, 'S'},
+			{"nosort", no_argument, NULL, 'N'},
 			{NULL, 0, NULL, 0}
 		};
 
@@ -1680,6 +1720,7 @@ static int cmd_subvol_list(int argc, char **argv)
 			}
 			break;
 		case 'S':
+			sort = 1;
 			ret = btrfs_list_parse_sort_string_v2(optarg,
 							   &comparer_set);
 			if (ret) {
@@ -1687,6 +1728,9 @@ static int cmd_subvol_list(int argc, char **argv)
 				goto out;
 			}
 			break;
+		case 'N':
+			no_sort = 1;
+			break;
 
 		default:
 			uerr = 1;
@@ -1717,6 +1761,12 @@ static int cmd_subvol_list(int argc, char **argv)
 		goto out;
 	}
 
+	if (sort && no_sort) {
+		ret = -1;
+		error("cannot use --sort with --nosort option");
+		goto out;
+	}
+
 	subvol = argv[optind];
 	fd = btrfs_open_dir(subvol, &dirstream, 1);
 	if (fd < 0) {
@@ -1750,7 +1800,7 @@ static int cmd_subvol_list(int argc, char **argv)
 
 	ret = btrfs_list_subvols_print_v2(fd, filter_set, comparer_set,
 			layout, is_list_all, absolute_path, follow_mount,
-			subvol, NULL);
+			no_sort, subvol, NULL);
 
 out:
 	close_file_or_dir(fd, dirstream);
-- 
2.14.4

[PATCH v2 13/20] btrfs-progs: sub list: Update -a option and remove meaningless filter

From: Misono Tomohiro <hidden>
Date: 2018-06-18 08:38:16

Currently, -a option add filter and change subvolume path as follows:
  - If a subvolume is a child of the specified path, nothing changes
  - otherwise, adds <FS_TREE> to head

This is rather meaningless, so let's remove this filter.

As a result, the behavior of -a option becomes the same as
default behavior of sub list in progs <= 4.17

[Example]
 $ mkfs.btrfs -f $DEV
 $ mount $DEV /mnt

 $ btrfs subvolume create /mnt/AAA
 $ btrfs subvolume create /mnt/AAA/BBB
 $ btrfs subvolume create /mnt/ZZZ

 $ btrfs subvolume list -a /mnt
 ID 256 gen 9 top level 5 path AAA
 ID 257 gen 9 top level 256 path AAA/BBB
 ID 258 gen 10 top level 5 path ZZZ

 ** output of progs <= 4.17
 $ btrfs subvolume list -a /mnt
 ID 256 gen 9 top level 5 path AAA
 ID 257 gen 9 top level 256 path <FS_TREE>/AAA/BBB
 ID 258 gen 10 top level 5 path ZZZ

Signed-off-by: Misono Tomohiro <redacted>
---
 Documentation/btrfs-subvolume.asciidoc |  6 ++++--
 cmds-subvolume.c                       | 35 ++++------------------------------
 2 files changed, 8 insertions(+), 33 deletions(-)
diff --git a/Documentation/btrfs-subvolume.asciidoc b/Documentation/btrfs-subvolume.asciidoc
index cd91385a..20fae1e1 100644
--- a/Documentation/btrfs-subvolume.asciidoc
+++ b/Documentation/btrfs-subvolume.asciidoc
@@ -118,8 +118,10 @@ Path filtering;;
 -o::::
 print only subvolumes below specified <path>.
 -a::::
-print all the subvolumes in the filesystem and distinguish between
-absolute and relative path with respect to the given <path>.
+print all the subvolumes in the filesystem, including subvolumes
+which cannot be accessed from current mount point.
+path to be shown is relative to the top-level subvolume
+(require root privileges).
 -f::::
 follow mounted subvolumes below <path> recursively and list them too
 (only if it is the same filesystem). If top-level subvolume is mounted
diff --git a/cmds-subvolume.c b/cmds-subvolume.c
index 802f5c5e..dab266aa 100644
--- a/cmds-subvolume.c
+++ b/cmds-subvolume.c
@@ -759,28 +759,6 @@ static int filter_topid_equal(struct listed_subvol *subvol, uint64_t data)
 	return subvol->info.parent_id == data;
 }
 
-static int filter_full_path(struct listed_subvol *subvol, uint64_t data)
-{
-	/*
-	 * This implements the same behavior as before the conversion to
-	 * libbtrfsutil, which is mostly nonsensical.
-	 */
-	if (subvol->info.parent_id != data) {
-		char *tmp;
-		int ret;
-
-		ret = asprintf(&tmp, "<FS_TREE>/%s", subvol->path);
-		if (ret == -1) {
-			error("out of memory");
-			exit(1);
-		}
-
-		free(subvol->path);
-		subvol->path = tmp;
-	}
-	return 1;
-}
-
 static int filter_by_parent(struct listed_subvol *subvol, uint64_t data)
 {
 	return !uuid_compare(subvol->info.parent_uuid,
@@ -798,7 +776,6 @@ static btrfs_list_filter_func_v2 all_filter_funcs[] = {
 	[BTRFS_LIST_FILTER_CGEN_LESS]		= filter_cgen_less,
 	[BTRFS_LIST_FILTER_CGEN_EQUAL]          = filter_cgen_equal,
 	[BTRFS_LIST_FILTER_TOPID_EQUAL]		= filter_topid_equal,
-	[BTRFS_LIST_FILTER_FULL_PATH]		= filter_full_path,
 	[BTRFS_LIST_FILTER_BY_PARENT]		= filter_by_parent,
 };
 
@@ -1574,9 +1551,9 @@ static const char * const cmd_subvol_list_usage[] = {
 	"",
 	"Path filtering:",
 	"-o           print only subvolumes below specified path",
-	"-a           print all the subvolumes in the filesystem and",
-	"             distinguish absolute and relative path with respect",
-	"             to the given <path> (require root privileges)",
+	"-a           print all the subvolumes in the filesystem.",
+	"             path to be shown is relative to the top-level",
+	"             subvolume (require root privileges)",
 	"-f           follow mounted subvolumes below the specified path",
 	"             and list them too (only if it is the same filesystem)",
 	"",
@@ -1783,11 +1760,7 @@ static int cmd_subvol_list(int argc, char **argv)
 	if (ret)
 		goto out;
 
-	if (is_list_all)
-		btrfs_list_setup_filter_v2(&filter_set,
-					BTRFS_LIST_FILTER_FULL_PATH,
-					top_id);
-	else if (is_only_in_path)
+	if (is_only_in_path)
 		btrfs_list_setup_filter_v2(&filter_set,
 					BTRFS_LIST_FILTER_TOPID_EQUAL,
 					top_id);
-- 
2.14.4

[PATCH v2 08/20] btrfs-progs: sub list: factor out main part of btrfs_list_subvols

From: Misono Tomohiro <hidden>
Date: 2018-06-18 08:38:17

No functional changes.
This is a preparation work for reworking "subvolume list".

Signed-off-by: Misono Tomohiro <redacted>
---
 cmds-subvolume.c | 50 ++++++++++++++++++++++++++++++--------------------
 1 file changed, 30 insertions(+), 20 deletions(-)
diff --git a/cmds-subvolume.c b/cmds-subvolume.c
index c54a8003..279a6e51 100644
--- a/cmds-subvolume.c
+++ b/cmds-subvolume.c
@@ -1125,22 +1125,15 @@ out:
 	return subvols;
 }
 
-static struct subvol_list *btrfs_list_subvols(int fd,
-					      struct btrfs_list_filter_set_v2 *filter_set)
+static void get_subvols_info(struct subvol_list **subvols,
+			     struct btrfs_list_filter_set_v2 *filter_set,
+			     int fd,
+			     size_t *capacity)
 {
-	struct subvol_list *subvols;
-	size_t capacity = 0;
 	struct btrfs_util_subvolume_iterator *iter;
 	enum btrfs_util_error err;
 	int ret = -1;
 
-	subvols = malloc(sizeof(*subvols));
-	if (!subvols) {
-		error("out of memory");
-		return NULL;
-	}
-	subvols->num = 0;
-
 	err = btrfs_util_create_subvolume_iterator_fd(fd,
 						      BTRFS_FS_TREE_OBJECTID, 0,
 						      &iter);
@@ -1168,11 +1161,11 @@ static struct subvol_list *btrfs_list_subvols(int fd,
 			continue;
 		}
 
-		if (subvols->num >= capacity) {
+		if ((*subvols)->num >= *capacity) {
 			struct subvol_list *new_subvols;
-			size_t new_capacity = max_t(size_t, 1, capacity * 2);
+			size_t new_capacity = max_t(size_t, 1, *capacity * 2);
 
-			new_subvols = realloc(subvols,
+			new_subvols = realloc(*subvols,
 					      sizeof(*new_subvols) +
 					      new_capacity *
 					      sizeof(new_subvols->subvols[0]));
@@ -1181,12 +1174,12 @@ static struct subvol_list *btrfs_list_subvols(int fd,
 				goto out;
 			}
 
-			subvols = new_subvols;
-			capacity = new_capacity;
+			*subvols = new_subvols;
+			*capacity = new_capacity;
 		}
 
-		subvols->subvols[subvols->num] = subvol;
-		subvols->num++;
+		(*subvols)->subvols[(*subvols)->num] = subvol;
+		(*subvols)->num++;
 	}
 
 	ret = 0;
@@ -1194,9 +1187,26 @@ out:
 	if (iter)
 		btrfs_util_destroy_subvolume_iterator(iter);
 	if (ret) {
-		free_subvol_list(subvols);
-		subvols = NULL;
+		free_subvol_list(*subvols);
+		*subvols = NULL;
+	}
+}
+
+static struct subvol_list *btrfs_list_subvols(int fd,
+					      struct btrfs_list_filter_set_v2 *filter_set)
+{
+	struct subvol_list *subvols;
+	size_t capacity = 0;
+
+	subvols = malloc(sizeof(*subvols));
+	if (!subvols) {
+		error("out of memory");
+		return NULL;
 	}
+	subvols->num = 0;
+
+	get_subvols_info(&subvols, filter_set, fd, &capacity);
+
 	return subvols;
 }
 
-- 
2.14.4

[PATCH v2 17/20] btrfs-progs: sub show: Allow non-privileged user to call "subvolume show"

From: Misono Tomohiro <hidden>
Date: 2018-06-18 08:38:17

Allow non-privileged user to call subvolume show if new ioctls
(BTRFS_IOC_GET_SUBVOL_INFO/BTRFS_IOC_GET_SUBVOL_ROOTREF,
BTRFS_IOC_INO_LOOKUP_USER, from kernel 4.18) are available.
Non-privileged user still cannot use -r or -u option.

The behavior for root user is the same as before.

There are some output differences between root and user:
  root ... subvolume path is from top-level subvolume
           list all snapshots in the fs (inc. non-accessible ones)
  user ... subvolume path is absolute path
           list snapshots under the mountpoint
	   (only to which the user has appropriate access right)

[Example]
 $ sudo mkfs.btrfs -f $DEV
 $ sudo mount $DEV /mnt

 $ sudo btrfs subvolume create /mnt/AAA
 $ sudo btrfs subvolume snapshot /mnt/AAA /mnt/snap1
 $ sudo btrfs subvolume snapshot /mnt/AAA /mnt/AAA/snap2

 $ sudo umount /mnt
 $ sudo mount -o subvol=AAA $DEV /mnt

 # root
 $ sudo btrfs subvolume show /mnt
 AAA
      Name:        AAA
      UUID:        15e80697-2ffb-0b4b-8e1e-e0873a7cf944
      ...
      Snapshot(s):
                   AAA/snap2
                   snap1

 # non-privileged user
 $ btrfs subvolume show /mnt
 /mnt
      Name:        AAA
      UUID:        15e80697-2ffb-0b4b-8e1e-e0873a7cf944
      ...
      Snapshot(s):
                   /mnt/snap2

Signed-off-by: Misono Tomohiro <redacted>
---
 Documentation/btrfs-subvolume.asciidoc |  11 +++-
 cmds-subvolume.c                       | 107 +++++++++++++++++++++++++++++----
 2 files changed, 105 insertions(+), 13 deletions(-)
diff --git a/Documentation/btrfs-subvolume.asciidoc b/Documentation/btrfs-subvolume.asciidoc
index 2db1d479..fc57e6bc 100644
--- a/Documentation/btrfs-subvolume.asciidoc
+++ b/Documentation/btrfs-subvolume.asciidoc
@@ -194,12 +194,19 @@ The id can be obtained from *btrfs subvolume list*, *btrfs subvolume show* or
 *show* [options] <path>|<mnt>::
 Show information of a given subvolume in the <path>.
 +
+This command had required root privileges. From kernel 4.18,
+non-privileged user can call this unless -r/-u option is not used.
+Note that for root, output path is relative to the top-level subvolume
+while absolute path is shown for non-privileged user.
+Also for root, snapshots filed lists all the snapshots in the fs while
+only snapshots under mount point are shown for non-privileged user.
++
 `Options`
 +
 -r|--rootid::::
-rootid of the subvolume.
+rootid of the subvolume (require root privileges).
 -u|--uuid:::
-UUID of the subvolume.
+UUID of the subvolume (require root privileges).
 
 +
 If no option is specified, subvolume information of <path> is shown,
diff --git a/cmds-subvolume.c b/cmds-subvolume.c
index ef39789a..4df418b0 100644
--- a/cmds-subvolume.c
+++ b/cmds-subvolume.c
@@ -2095,8 +2095,12 @@ static int cmd_subvol_find_new(int argc, char **argv)
 static const char * const cmd_subvol_show_usage[] = {
 	"btrfs subvolume show [options] <subvol-path>|<mnt>",
 	"Show more information about the subvolume",
-	"-r|--rootid   rootid of the subvolume",
-	"-u|--uuid     uuid of the subvolume",
+	"",
+	"This command had required root privileges. From kernel 4.18,",
+	"non-privileged user can call this unless -r/-u option is not used.",
+	"",
+	"-r|--rootid   rootid of the subvolume (require root privileges)",
+	"-u|--uuid     uuid of the subvolume   (require root privileges)",
 	"",
 	"If no option is specified, <subvol-path> will be shown, otherwise",
 	"the rootid or uuid are resolved relative to the <mnt> path.",
@@ -2109,8 +2113,10 @@ static int cmd_subvol_show(int argc, char **argv)
 	char uuidparse[BTRFS_UUID_UNPARSED_SIZE];
 	char *fullpath = NULL;
 	int fd = -1;
+	int fd_mnt = -1;
 	int ret = 1;
 	DIR *dirstream1 = NULL;
+	DIR *dirstream_mnt = NULL;
 	int by_rootid = 0;
 	int by_uuid = 0;
 	u64 rootid_arg = 0;
@@ -2118,7 +2124,10 @@ static int cmd_subvol_show(int argc, char **argv)
 	struct btrfs_util_subvolume_iterator *iter;
 	struct btrfs_util_subvolume_info subvol;
 	char *subvol_path = NULL;
+	char *subvol_name = NULL;
+	char *mount_point = NULL;
 	enum btrfs_util_error err;
+	bool root;
 
 	while (1) {
 		int c;
@@ -2155,6 +2164,12 @@ static int cmd_subvol_show(int argc, char **argv)
 		usage(cmd_subvol_show_usage);
 	}
 
+	root = is_root();
+	if (!root && (by_rootid || by_uuid)) {
+		error("only root can use -r or -u options");
+		return -1;
+	}
+
 	fullpath = realpath(argv[optind], NULL);
 	if (!fullpath) {
 		error("cannot find real path for '%s': %m", argv[optind]);
@@ -2209,19 +2224,53 @@ static int cmd_subvol_show(int argc, char **argv)
 			goto out;
 		}
 
-		err = btrfs_util_subvolume_path_fd(fd, subvol.id, &subvol_path);
-		if (err) {
-			error_btrfs_util(err);
-			goto out;
+		if (root) {
+			/* Construct path relative to top-level subvolume */
+			err = btrfs_util_subvolume_path_fd(fd, subvol.id,
+								&subvol_path);
+			if (err) {
+				error_btrfs_util(err);
+				goto out;
+			}
+			subvol_name = strdup(basename(subvol_path));
+		} else {
+			/* Show absolute path */
+			subvol_path = strdup(fullpath);
+
+			ret = find_mount_root(fullpath, &mount_point);
+			if (ret < 0) {
+				error("cannot get mount point");
+				goto out;
+			}
+			fd_mnt = open_file_or_dir(mount_point, &dirstream_mnt);
+			if (fd_mnt < 0) {
+				error("cannot open mount point");
+				goto out;
+			}
+			/* Get real name if the path is mount point */
+			if (strlen(fullpath) == strlen(mount_point)) {
+				struct btrfs_ioctl_get_subvol_info_args arg;
+
+				ret = ioctl(fd_mnt, BTRFS_IOC_GET_SUBVOL_INFO,
+							&arg);
+				if (ret < 0) {
+					error("cannot get subvolume info");
+					goto out;
+				}
+				subvol_name = strdup(arg.name);
+			} else {
+				subvol_name = strdup(basename(subvol_path));
+			}
 		}
 
 	}
 
 	/* print the info */
-	printf("%s\n", subvol.id == BTRFS_FS_TREE_OBJECTID ? "/" : subvol_path);
+	printf("%s\n", (subvol.id == BTRFS_FS_TREE_OBJECTID && root) ?
+			"/" : subvol_path);
 	printf("\tName: \t\t\t%s\n",
 	       (subvol.id == BTRFS_FS_TREE_OBJECTID ? "<FS_TREE>" :
-		basename(subvol_path)));
+					subvol_name));
 
 	if (uuid_is_null(subvol.uuid))
 		strcpy(uuidparse, "-");
@@ -2264,9 +2313,18 @@ static int cmd_subvol_show(int argc, char **argv)
 	/* print the snapshots of the given subvol if any*/
 	printf("\tSnapshot(s):\n");
 
-	err = btrfs_util_create_subvolume_iterator_fd(fd,
-						      BTRFS_FS_TREE_OBJECTID, 0,
-						      &iter);
+	/*
+	 * For root, show all snapshots in the filesystem.
+	 * For non-privileged user, show all snapshots under mount point.
+	 */
+	if (root)
+		err = btrfs_util_create_subvolume_iterator_fd(fd,
+					      BTRFS_FS_TREE_OBJECTID, 0,
+					      &iter);
+	else
+		err = btrfs_util_create_subvolume_iterator_fd(fd_mnt,
+					      0, 0,
+					      &iter);
 
 	for (;;) {
 		struct btrfs_util_subvolume_info subvol2;
@@ -2278,9 +2336,33 @@ static int cmd_subvol_show(int argc, char **argv)
 		} else if (err) {
 			error_btrfs_util(err);
 			btrfs_util_destroy_subvolume_iterator(iter);
+			ret = -1;
 			goto out;
 		}
 
+		if (!root) {
+			/* Make path absolute */
+			char *temp = malloc(strlen(mount_point) +
+						   strlen(path) + 2);
+
+			if (!temp) {
+				error("out of memory");
+				ret = -1;
+				goto out;
+			}
+
+			strcpy(temp, mount_point);
+			if (strlen(mount_point) == 1) {
+				strcpy(temp + 1, path);
+			} else {
+				temp[strlen(mount_point)] = '/';
+				strcpy(temp + strlen(mount_point) + 1, path);
+			}
+
+			free(path);
+			path = temp;
+		}
+
 		if (uuid_compare(subvol2.parent_uuid, subvol.uuid) == 0)
 			printf("\t\t\t\t%s\n", path);
 
@@ -2291,8 +2373,11 @@ static int cmd_subvol_show(int argc, char **argv)
 	ret = 0;
 out:
 	free(subvol_path);
+	free(subvol_name);
 	close_file_or_dir(fd, dirstream1);
+	close_file_or_dir(fd_mnt, dirstream_mnt);
 	free(fullpath);
+	free(mount_point);
 	return !!ret;
 }
 
-- 
2.14.4

[PATCH v2 19/20] btrfs-porgs: test: Add cli-test/009 to check subvolume list for both root and normal user

From: Misono Tomohiro <hidden>
Date: 2018-06-18 08:38:17

Signed-off-by: Misono Tomohiro <redacted>
---
 tests/cli-tests/009-subvolume-list/test.sh | 134 +++++++++++++++++++++++++++++
 1 file changed, 134 insertions(+)
 create mode 100755 tests/cli-tests/009-subvolume-list/test.sh
diff --git a/tests/cli-tests/009-subvolume-list/test.sh b/tests/cli-tests/009-subvolume-list/test.sh
new file mode 100755
index 00000000..bf6c31d6
--- /dev/null
+++ b/tests/cli-tests/009-subvolume-list/test.sh
@@ -0,0 +1,134 @@
+#!/bin/bash
+# test for "subvolume list" both for root and normal user
+
+source "$TEST_TOP/common"
+
+check_testuser
+check_prereq mkfs.btrfs
+check_prereq btrfs
+
+setup_root_helper
+prepare_test_dev
+
+
+# test if the ids returned by "sub list" match expected ids
+# $1  ... indicate run as root or test user
+# $2  ... PATH to be specified by sub list command
+# $3~ ... expected return ids
+test_list()
+{
+	local SUDO
+	if [ $1 -eq 1 ]; then
+		SUDO=$SUDO_HELPER
+	else
+		SUDO="sudo -u progs-test"
+	fi
+
+	result=$(run_check_stdout $SUDO "$TOP/btrfs" subvolume list "$2" | \
+		awk '{print $2}' | xargs | sort -n)
+
+	shift
+	shift
+	expected=($(echo "$@" | tr " " "\n" | sort -n))
+	expected=$(IFS=" "; echo "${expected[*]}")
+
+	if [ "$result" != "$expected" ]; then
+		echo "result  : $result"
+		echo "expected: $expected"
+		_fail "ids returned by sub list does not match expected ids"
+	fi
+}
+
+run_check $SUDO_HELPER "$TOP/mkfs.btrfs" -f "$TEST_DEV"
+run_check_mount_test_dev
+cd "$TEST_MNT"
+
+# create subvolumes and directories and make some non-readable
+# by user 'progs-test'
+run_check $SUDO_HELPER "$TOP/btrfs" subvolume create sub1
+run_check $SUDO_HELPER "$TOP/btrfs" subvolume create sub1/subsub1
+run_check $SUDO_HELPER mkdir sub1/dir
+
+run_check $SUDO_HELPER "$TOP/btrfs" subvolume create sub2
+run_check $SUDO_HELPER mkdir -p sub2/dir/dirdir
+run_check $SUDO_HELPER "$TOP/btrfs" subvolume create sub2/dir/subsub2
+run_check $SUDO_HELPER "$TOP/btrfs" subvolume create sub2/dir/dirdir/subsubX
+
+run_check $SUDO_HELPER "$TOP/btrfs" subvolume create sub3
+run_check $SUDO_HELPER "$TOP/btrfs" subvolume create sub3/subsub3
+run_check $SUDO_HELPER mkdir sub3/dir
+run_check $SUDO_HELPER "$TOP/btrfs" subvolume create sub3/dir/subsubY
+run_check $SUDO_HELPER chmod o-r sub3
+
+run_check $SUDO_HELPER "$TOP/btrfs" subvolume create sub4
+run_check $SUDO_HELPER "$TOP/btrfs" subvolume create sub4/subsub4
+run_check $SUDO_HELPER mkdir sub4/dir
+run_check $SUDO_HELPER "$TOP/btrfs" subvolume create sub4/dir/subsubZ
+run_check $SUDO_HELPER setfacl -m u:progs-test:- sub4/dir
+
+run_check $SUDO_HELPER touch "file"
+
+# expected result for root at mount point:
+#
+# ID 256 gen 8 top level 5 path sub1
+# ID 258 gen 7 top level 256 path sub1/subsub1
+# ID 259 gen 10 top level 5 path sub2
+# ID 260 gen 9 top level 259 path sub2/dir/subsub2
+# ID 261 gen 10 top level 259 path sub2/dir/dirdir/subsubX
+# ID 262 gen 14 top level 5 path sub3
+# ID 263 gen 12 top level 262 path sub3/subsub3
+# ID 264 gen 13 top level 262 path sub3/dir/subsubY
+# ID 265 gen 17 top level 5 path sub4
+# ID 266 gen 15 top level 265 path sub4/subsub4
+# ID 267 gen 16 top level 265 path sub4/dir/subsubZ
+
+# check for root for both absolute/relative path
+all=(256 258 259 260 261 262 263 264 265 266 267)
+test_list 1 "$TEST_MNT" "${all[@]}"
+test_list 1 "$TEST_MNT/sub1" "256 258"
+test_list 1 "$TEST_MNT/sub1/dir" ""
+test_list 1 "$TEST_MNT/sub2" "259 260 261"
+test_list 1 "$TEST_MNT/sub2/dir" "260 261"
+test_list 1 "$TEST_MNT/sub3" "262 263 264"
+test_list 1 "$TEST_MNT/sub4" "265 266 267"
+run_mustfail "should fail for file" \
+	$SUDO_HELPER "$TOP/btrfs" subvolume list "$TEST_MNT/file"
+
+test_list 1 "." "${all[@]}"
+test_list 1 "sub1" "256 258"
+test_list 1 "sub1/dir" ""
+test_list 1 "sub2" "259 260 261"
+test_list 1 "sub2/dir" "260 261"
+test_list 1 "sub3" "262 263 264"
+test_list 1 "sub4" "265 266 267"
+run_mustfail "should fail for file" \
+	$SUDO_HELPER "$TOP/btrfs" subvolume list "file"
+
+# check for normal user for both absolute/relative path
+test_list 0 "$TEST_MNT" "256 258 259 260 261 265 266"
+test_list 0 "$TEST_MNT/sub1" "256 258"
+test_list 0 "$TEST_MNT/sub1/dir" ""
+test_list 0 "$TEST_MNT/sub2" "259 260 261"
+test_list 0 "$TEST_MNT/sub2/dir" "260 261"
+run_mustfail "should raise permission error" \
+	sudo -u progs-test "$TOP/btrfs" subvolume list "$TEST_MNT/sub3"
+test_list 0 "$TEST_MNT/sub4" "265 266"
+run_mustfail "should raise permission error" \
+	sudo -u progs-test "$TOP/btrfs" subvolume list "$TEST_MNT/sub4/dir"
+run_mustfail "should fail for file" \
+	sudo -u progs-test "$TOP/btrfs" subvolume list "$TEST_MNT/file"
+
+test_list 0 "." "256 258 259 260 261 265 266"
+test_list 0 "sub1/dir" ""
+test_list 0 "sub2" "259 260 261"
+test_list 0 "sub2/dir" "260 261"
+run_mustfail "should raise permission error" \
+	sudo -u progs-test "$TOP/btrfs" subvolume list "sub3"
+test_list 0 "sub4" "265 266"
+run_mustfail "should raise permission error" \
+	sudo -u progs-test "$TOP/btrfs" subvolume list "sub4/dir"
+run_mustfail "should fail for file" \
+	sudo -u progs-test "$TOP/btrfs" subvolume list "file"
+
+cd ..
+run_check_umount_test_dev
-- 
2.14.4

[PATCH v2 20/20] btrfs-progs: test: Add cli-test/010 to check "subvolume list -f" option

From: Misono Tomohiro <hidden>
Date: 2018-06-18 08:38:17

Signed-off-by: Misono Tomohiro <redacted>
---
 tests/cli-tests/010-subvolume-list-follow/test.sh | 86 +++++++++++++++++++++++
 1 file changed, 86 insertions(+)
 create mode 100755 tests/cli-tests/010-subvolume-list-follow/test.sh
diff --git a/tests/cli-tests/010-subvolume-list-follow/test.sh b/tests/cli-tests/010-subvolume-list-follow/test.sh
new file mode 100755
index 00000000..8fb746c6
--- /dev/null
+++ b/tests/cli-tests/010-subvolume-list-follow/test.sh
@@ -0,0 +1,86 @@
+#!/bin/bash
+# test for "subvolume list -f"
+
+source "$TEST_TOP/common"
+
+check_prereq mkfs.btrfs
+check_prereq btrfs
+
+setup_root_helper
+setup_loopdevs 2
+prepare_loopdevs
+dev1=${loopdevs[1]}
+dev2=${loopdevs[2]}
+
+# test if the ids returned by "sub list" match expected ids
+# $1  ... option for subvolume list
+# $2  ... PATH to be specified by sub list command
+# $3~ ... expected return ids
+test_list()
+{
+	result=$(run_check_stdout $SUDO "$TOP/btrfs" subvolume list $1 "$2" | \
+		awk '{print $2}' | xargs | sort -n)
+
+	shift
+	shift
+	expected=($(echo "$@" | tr " " "\n" | sort -n))
+	expected=$(IFS=" "; echo "${expected[*]}")
+
+	if [ "$result" != "$expected" ]; then
+		echo "result  : $result"
+		echo "expected: $expected"
+		_fail "ids returned by sub list does not match expected ids"
+	fi
+}
+
+run_check $SUDO_HELPER "$TOP/mkfs.btrfs" -f "$dev1"
+run_check $SUDO_HELPER "$TOP/mkfs.btrfs" -f "$dev2"
+
+run_check $SUDO_HELPER mount "$dev1" "$TEST_MNT"
+cd "$TEST_MNT"
+
+# Create some subvolumes and directories
+# <FS_TREE> (id 5)
+#   |- AAA (id 256)
+#   |   |- top
+#   |   |- bbb
+#   |   -- ccc
+#   |
+#   |- BBB (id 258)
+#   -- CCC (id 259)
+run_check $SUDO_HELPER "$TOP/btrfs" subvolume create AAA
+run_check $SUDO_HELPER "$TOP/btrfs" subvolume create BBB
+run_check $SUDO_HELPER "$TOP/btrfs" subvolume create CCC
+run_check $SUDO_HELPER mkdir AAA/top
+run_check $SUDO_HELPER mkdir AAA/bbb
+run_check $SUDO_HELPER mkdir AAA/ccc
+
+test_list "" "." "256 258 259"
+test_list "-f" "." "256 258 259"
+cd ..
+run_check $SUDO_HELPER umount "$TEST_MNT"
+
+# Mount as follows:
+#
+# "TEST_MNT" (AAA)
+#   |- top (FS_TREE)
+#   |   |- AAA
+#   |   |- BBB
+#   |   -- CCC
+#   |
+#   |- bbb (BBB)
+#   -- ccc (CCC)
+run_check $SUDO_HELPER mount -o subvol=AAA "$dev1" "$TEST_MNT"
+run_check $SUDO_HELPER mount "$dev1" "$TEST_MNT/top"
+run_check $SUDO_HELPER mount -o subvol=BBB "$dev1" "$TEST_MNT/bbb"
+run_check $SUDO_HELPER mount -o subvol=CCC "$dev1" "$TEST_MNT/ccc"
+
+cd "$TEST_MNT"
+test_list "" "." "256"
+# With -f option, subvolume AAA/BBB/CCC will be counted twice.
+# Also, it will list FS_TREE (5) if it is mounted below the specified path.
+test_list "-f" "." "5 256 256 258 258 259 259"
+
+cd ..
+run_check $SUDO_HELPER umount -R "$TEST_MNT"
+cleanup_loopdevs
-- 
2.14.4

[PATCH v2 09/20] btrfs-progs: sub list: Change the default behavior of "subvolume list" and allow non-privileged user to call it

From: Misono Tomohiro <hidden>
Date: 2018-06-18 08:38:18

Change the default behavior of "subvolume list" and allow non-privileged
user to call it as well.
From this commit, by default it only lists subvolumes under the specified
path (incl. the path itself except top-level subvolume). Also, if kernel
supports new ioctls (BTRFS_IOC_GET_SUBVOL_INFO/BTRFS_IOC_GET_SUBVOL_ROOTREF
and BTRFS_IOC_INO_LOOKUP_USER, which are avilable from 4.18),
  - the specified path can be non-subvolume directory.
  - non-privileged user can also call it (subvolumes which the user
    does not have access right will be skiped).

Note that root user can list all the subvolume in the fs with -a option.

[Example]
 $ mkfs.btrfs -f $DEV
 $ mount $DEV /mnt

 $ btrfs subvolume create /mnt/AAA
 $ btrfs subvolume create /mnt/AAA/BBB
 $ mkdir /mnt/AAA/BBB/dir
 $ btrfs subvolume create /mnt/AAA/BBB/dir/CCC
 $ btrfs subvolume create /mnt/ZZZ

 $ umount /mnt
 $ mount -o subvol=AAA $DEV /mnt

 $ btrfs subvolume list /mnt
 ID 256 gen 11 top level 5 path .
 ID 257 gen 8 top level 256 path BBB
 ID 258 gen 8 top level 257 path BBB/dir/CCC

 $ btrfs subvolume list /mnt/BBB
 ID 257 gen 8 top level 256 path .
 ID 258 gen 8 top level 257 path dir/CCC

 $ btrfs subvolume list /mnt/BBB/dir
 ID 258 gen 8 top level 257 path CCC

 ** output of progs <= 4.17
 $ mount -o subvol=AAA $DEV /mnt
 $ btrfs subvolume list /mnt
 ID 256 gen 11 top level 5 path AAA
 ID 257 gen 8 top level 256 path BBB
 ID 258 gen 8 top level 257 path BBB/dir/CCC
 ID 259 gen 11 top level 256 path ZZZ

Signed-off-by: Misono Tomohiro <redacted>
---
 Documentation/btrfs-subvolume.asciidoc |   8 +-
 cmds-subvolume.c                       | 144 +++++++++++++++++++++++++--------
 2 files changed, 119 insertions(+), 33 deletions(-)
diff --git a/Documentation/btrfs-subvolume.asciidoc b/Documentation/btrfs-subvolume.asciidoc
index f3eb4e26..99fff977 100644
--- a/Documentation/btrfs-subvolume.asciidoc
+++ b/Documentation/btrfs-subvolume.asciidoc
@@ -95,6 +95,12 @@ The output format is similar to *subvolume list* command.
 
 *list* [options] [-G [\+|-]<value>] [-C [+|-]<value>] [--sort=rootid,gen,ogen,path] <path>::
 List the subvolumes present in the filesystem <path>.
+By default, this only lists the subvolumes under <path>,
+including <path> itself (except top-level subvolume).
++
+This command had required root privileges. From kernel 4.18,
+non privileged user can call this too. Also from kernel 4.18,
+It is possible to specify non-subvolume directory as <path>.
 +
 For every subvolume the following information is shown by default:
 +
@@ -102,7 +108,7 @@ ID <ID> gen <generation> top level <ID> path <path>
 +
 where ID is subvolume's id, gen is an internal counter which is updated
 every transaction, top level is the same as parent subvolume's id, and
-path is the relative path of the subvolume to the top level subvolume.
+path is the relative path of the subvolume to the specified path.
 The subvolume's ID may be used by the subvolume set-default command,
 or at mount time via the subvolid= option.
 +
diff --git a/cmds-subvolume.c b/cmds-subvolume.c
index 279a6e51..23596c17 100644
--- a/cmds-subvolume.c
+++ b/cmds-subvolume.c
@@ -1125,9 +1125,37 @@ out:
 	return subvols;
 }
 
+static int add_subvol(struct subvol_list **subvols,
+		      struct listed_subvol *subvol,
+		      size_t *capacity)
+{
+	if ((*subvols)->num >= *capacity) {
+		struct subvol_list *new_subvols;
+		size_t new_capacity = max_t(size_t, 1, *capacity * 2);
+
+		new_subvols = realloc(*subvols,
+				      sizeof(*new_subvols) +
+				      new_capacity *
+				      sizeof(new_subvols->subvols[0]));
+		if (!new_subvols) {
+			error("out of memory");
+			return -1;
+		}
+
+		*subvols = new_subvols;
+		*capacity = new_capacity;
+	}
+
+	(*subvols)->subvols[(*subvols)->num] = *subvol;
+	(*subvols)->num++;
+
+	return 0;
+}
+
 static void get_subvols_info(struct subvol_list **subvols,
 			     struct btrfs_list_filter_set_v2 *filter_set,
 			     int fd,
+			     int tree_id,
 			     size_t *capacity)
 {
 	struct btrfs_util_subvolume_iterator *iter;
@@ -1135,7 +1163,7 @@ static void get_subvols_info(struct subvol_list **subvols,
 	int ret = -1;
 
 	err = btrfs_util_create_subvolume_iterator_fd(fd,
-						      BTRFS_FS_TREE_OBJECTID, 0,
+						      tree_id, 0,
 						      &iter);
 	if (err) {
 		iter = NULL;
@@ -1143,6 +1171,52 @@ static void get_subvols_info(struct subvol_list **subvols,
 		goto out;
 	}
 
+	/*
+	 * Subvolume iterator does not include the information of the
+	 * specified path/fd. So, add it first.
+	 */
+	if (!tree_id) {
+		uint64_t id;
+		struct listed_subvol subvol;
+
+		err = btrfs_util_is_subvolume_fd(fd);
+		if (err != BTRFS_UTIL_OK) {
+			if (err == BTRFS_UTIL_ERROR_NOT_SUBVOLUME) {
+				ret = 0;
+				goto skip;
+			} else {
+				ret = -1;
+				goto out;
+			}
+		}
+		err = btrfs_util_subvolume_id_fd(fd, &id);
+		if (err) {
+			ret = -1;
+			goto out;
+		}
+		if (id == BTRFS_FS_TREE_OBJECTID) {
+			/* Skip top level subvolume */
+			ret = 0;
+			goto skip;
+		}
+
+		err = btrfs_util_subvolume_info_fd(fd, 0, &subvol.info);
+		if (err) {
+			ret = -1;
+			goto out;
+		}
+
+		subvol.path = strdup(".");
+		if (!filters_match(&subvol, filter_set)) {
+			free(subvol.path);
+		} else {
+			ret = add_subvol(subvols, &subvol, capacity);
+			if (ret)
+				goto out;
+		}
+	}
+
+skip:
 	for (;;) {
 		struct listed_subvol subvol;
 
@@ -1153,33 +1227,17 @@ static void get_subvols_info(struct subvol_list **subvols,
 			break;
 		} else if (err) {
 			error_btrfs_util(err);
+			ret = -1;
 			goto out;
 		}
 
 		if (!filters_match(&subvol, filter_set)) {
 			free(subvol.path);
-			continue;
-		}
-
-		if ((*subvols)->num >= *capacity) {
-			struct subvol_list *new_subvols;
-			size_t new_capacity = max_t(size_t, 1, *capacity * 2);
-
-			new_subvols = realloc(*subvols,
-					      sizeof(*new_subvols) +
-					      new_capacity *
-					      sizeof(new_subvols->subvols[0]));
-			if (!new_subvols) {
-				error("out of memory");
+		} else {
+			ret = add_subvol(subvols, &subvol, capacity);
+			if (ret)
 				goto out;
-			}
-
-			*subvols = new_subvols;
-			*capacity = new_capacity;
 		}
-
-		(*subvols)->subvols[(*subvols)->num] = subvol;
-		(*subvols)->num++;
 	}
 
 	ret = 0;
@@ -1193,6 +1251,8 @@ out:
 }
 
 static struct subvol_list *btrfs_list_subvols(int fd,
+					      int is_list_all,
+					      const char *path,
 					      struct btrfs_list_filter_set_v2 *filter_set)
 {
 	struct subvol_list *subvols;
@@ -1205,7 +1265,11 @@ static struct subvol_list *btrfs_list_subvols(int fd,
 	}
 	subvols->num = 0;
 
-	get_subvols_info(&subvols, filter_set, fd, &capacity);
+	if (is_list_all)
+		get_subvols_info(&subvols, filter_set, fd,
+				BTRFS_FS_TREE_OBJECTID, &capacity);
+	else
+		get_subvols_info(&subvols, filter_set, fd, 0, &capacity);
 
 	return subvols;
 }
@@ -1214,20 +1278,16 @@ static int btrfs_list_subvols_print_v2(int fd,
 				    struct btrfs_list_filter_set_v2 *filter_set,
 				    struct btrfs_list_comparer_set_v2 *comp_set,
 				    enum btrfs_list_layout layout,
-				    int full_path, const char *raw_prefix)
+				    int is_list_all,
+				    const char *path,
+				    const char *raw_prefix)
 {
 	struct subvol_list *subvols;
 
-	/*
-	 * full_path hasn't done anything since 4f5ebb3ef553 ("Btrfs-progs: fix
-	 * to make list specified directory's subvolumes work"). See
-	 * https://www.spinics.net/lists/linux-btrfs/msg69820.html
-	 */
-
 	if (filter_set->only_deleted)
 		subvols = btrfs_list_deleted_subvols(fd, filter_set);
 	else
-		subvols = btrfs_list_subvols(fd, filter_set);
+		subvols = btrfs_list_subvols(fd, is_list_all, path, filter_set);
 	if (!subvols)
 		return -1;
 
@@ -1323,6 +1383,14 @@ static int btrfs_list_parse_filter_string_v2(char *opt_arg,
 	return 0;
 }
 
+static bool is_root(void)
+{
+	uid_t uid;
+
+	uid = geteuid();
+	return (uid == 0);
+}
+
 /*
  * Naming of options:
  * - uppercase for filters and sort options
@@ -1331,12 +1399,18 @@ static int btrfs_list_parse_filter_string_v2(char *opt_arg,
 static const char * const cmd_subvol_list_usage[] = {
 	"btrfs subvolume list [options] <path>",
 	"List subvolumes and snapshots in the filesystem.",
+	"By default, this only lists the subvolumes under <path>,",
+	"including <path> itself (except top-level subvolume).",
+	"",
+	"This command had required root privileges. From kernel 4.18,",
+	"non privileged user can call this too. Also from kernel 4.18,",
+	"It is possible to specify non-subvolume directory as <path>.",
 	"",
 	"Path filtering:",
 	"-o           print only subvolumes below specified path",
 	"-a           print all the subvolumes in the filesystem and",
 	"             distinguish absolute and relative path with respect",
-	"             to the given <path>",
+	"             to the given <path> (require root privileges)",
 	"",
 	"Field selection:",
 	"-p           print parent ID",
@@ -1481,6 +1555,12 @@ static int cmd_subvol_list(int argc, char **argv)
 		goto out;
 	}
 
+	if (is_list_all && !is_root()) {
+		ret = -1;
+		error("only root can use -a option");
+		goto out;
+	}
+
 	subvol = argv[optind];
 	fd = btrfs_open_dir(subvol, &dirstream, 1);
 	if (fd < 0) {
@@ -1513,7 +1593,7 @@ static int cmd_subvol_list(int argc, char **argv)
 	btrfs_list_setup_print_column_v2(BTRFS_LIST_PATH);
 
 	ret = btrfs_list_subvols_print_v2(fd, filter_set, comparer_set,
-			layout, !is_list_all && !is_only_in_path, NULL);
+			layout, is_list_all, subvol, NULL);
 
 out:
 	close_file_or_dir(fd, dirstream);
-- 
2.14.4

[PATCH v2 11/20] btrfs-progs: sub list: Add -f option to follow mounted subvolumes below the path

From: Misono Tomohiro <hidden>
Date: 2018-06-18 08:38:18

Add -f option to follow mounted subvolumes below the specified path, only
if it is the same filesystem.

[Example]
 $ mkfs.btrfs -f $DEV
 $ mkfs.btrfs -f $DEV2
 $ mount $DEV /mnt

 $ btrfs subvolume create /mnt/AAA
 $ btrfs subvolume create /mnt/BBB
 $ btrfs subvolume create /mnt/CCC
 $ mkdir /mnt/AAA/bbb
 $ mkdir /mnt/AAA/ccc
 $ mkdir /mnt/AAA/other

 $ umount /mnt
 $ mount -o subvol=AAA $DEV /mnt
 $ mount -o subvol=BBB $DEV /mnt/bbb
 $ mount -o subvol=CCC $DEV /mnt/ccc
 $ mount -o $DEV2 /mnt/other

 $ btrfs subvolume list /mnt
 ID 256 gen 9 top level 5 path .

 $ btrfs subvolume list -f /mnt
 ID 256 gen 9 top level 5 path .
 ID 258 gen 7 top level 5 path bbb
 ID 259 gen 8 top level 5 path ccc

Note that this option lists top-level subvolume if it is mounted in the
way.

Signed-off-by: Misono Tomohiro <redacted>
---
 Documentation/btrfs-subvolume.asciidoc |   4 ++
 cmds-subvolume.c                       | 116 +++++++++++++++++++++++++++++++--
 2 files changed, 113 insertions(+), 7 deletions(-)
diff --git a/Documentation/btrfs-subvolume.asciidoc b/Documentation/btrfs-subvolume.asciidoc
index fec4b769..b2461398 100644
--- a/Documentation/btrfs-subvolume.asciidoc
+++ b/Documentation/btrfs-subvolume.asciidoc
@@ -120,6 +120,10 @@ print only subvolumes below specified <path>.
 -a::::
 print all the subvolumes in the filesystem and distinguish between
 absolute and relative path with respect to the given <path>.
+-f::::
+follow mounted subvolumes below <path> recursively and list them too
+(only if it is the same filesystem). If top-level subvolume is mounted
+in the way, it is also listed.
 
 Field selection;;
 -p::::
diff --git a/cmds-subvolume.c b/cmds-subvolume.c
index ea341d50..4ebe0377 100644
--- a/cmds-subvolume.c
+++ b/cmds-subvolume.c
@@ -28,6 +28,7 @@
 #include <getopt.h>
 #include <uuid/uuid.h>
 #include <linux/magic.h>
+#include <mntent.h>
 
 #include <btrfsutil.h>
 
@@ -1157,7 +1158,8 @@ static void get_subvols_info(struct subvol_list **subvols,
 			     int fd,
 			     int tree_id,
 			     size_t *capacity,
-			     const char *prefix)
+			     const char *prefix,
+			     int show_top)
 {
 	struct btrfs_util_subvolume_iterator *iter;
 	enum btrfs_util_error err;
@@ -1195,7 +1197,7 @@ static void get_subvols_info(struct subvol_list **subvols,
 			ret = -1;
 			goto out;
 		}
-		if (id == BTRFS_FS_TREE_OBJECTID) {
+		if (!show_top && id == BTRFS_FS_TREE_OBJECTID) {
 			/* Skip top level subvolume */
 			ret = 0;
 			goto skip;
@@ -1280,6 +1282,7 @@ out:
 static struct subvol_list *btrfs_list_subvols(int fd,
 					      int is_list_all,
 					      int absolute_path,
+					      int follow_mount,
 					      const char *path,
 					      struct btrfs_list_filter_set_v2 *filter_set)
 {
@@ -1295,7 +1298,8 @@ static struct subvol_list *btrfs_list_subvols(int fd,
 
 	if (is_list_all) {
 		get_subvols_info(&subvols, filter_set, fd,
-				BTRFS_FS_TREE_OBJECTID, &capacity, NULL);
+				BTRFS_FS_TREE_OBJECTID, &capacity, NULL,
+				false);
 	} else {
 		char *fullpath;
 
@@ -1307,8 +1311,92 @@ static struct subvol_list *btrfs_list_subvols(int fd,
 		}
 
 		get_subvols_info(&subvols, filter_set, fd, 0, &capacity,
-				(absolute_path ? fullpath : NULL));
+				(absolute_path ? fullpath : NULL), false);
 
+		if (subvols == NULL) {
+			free(fullpath);
+			return NULL;
+		}
+
+		/* Follow mounted subvolumes below @path */
+		if (follow_mount) {
+			struct mntent *mnt;
+			FILE *f;
+			DIR *dirstream;
+			u8 fsid[BTRFS_FSID_SIZE];
+			u8 fsid2[BTRFS_FSID_SIZE];
+			char *c;
+			int fd2;
+			int ret;
+
+			ret = get_fsid(path, fsid, 0);
+			if (ret < 0) {
+				error("failed to get fsid: %m");
+				free(fullpath);
+				free_subvol_list(subvols);
+				return NULL;
+			}
+
+			f = setmntent("/proc/self/mounts", "r");
+			if (f == NULL) {
+				error("failed to read mount entry: %m");
+				free(fullpath);
+				free_subvol_list(subvols);
+				return NULL;
+			}
+
+			/* Iterate for each mount entry */
+			while ((mnt = getmntent(f)) != NULL) {
+				if (strcmp(mnt->mnt_type, "btrfs"))
+					continue;
+
+				if (!strcmp(mnt->mnt_dir, fullpath))
+					continue;
+
+				c = strstr(mnt->mnt_dir, fullpath);
+				if (c != mnt->mnt_dir)
+					continue;
+
+				/* If fsid is different, skip it */
+				ret = get_fsid(mnt->mnt_dir, fsid2, 1);
+				if (ret < 0) {
+					/*
+					 * ENOENT may happen when mount is
+					 * stacked
+					 */
+					if (errno == EACCES || errno == ENOENT)
+						continue;
+					error("failed to get fsid: %m");
+					free(fullpath);
+					free_subvol_list(subvols);
+					return NULL;
+				}
+				if (uuid_compare(fsid, fsid2))
+					continue;
+
+				fd2 = btrfs_open_dir(mnt->mnt_dir,
+						     &dirstream, 1);
+				if (fd2 < 0) {
+					error("cannot open '%s': %m",
+							mnt->mnt_dir);
+					free(fullpath);
+					free_subvol_list(subvols);
+					return NULL;
+				}
+				get_subvols_info(&subvols, filter_set,
+						fd2, 0, &capacity,
+						(absolute_path ? mnt->mnt_dir :
+							(strlen(fullpath) == 1 ?
+							 mnt->mnt_dir + 1 :
+							 mnt->mnt_dir + strlen(fullpath) + 1)),
+						true);
+				close_file_or_dir(fd2, dirstream);
+				if (subvols == NULL) {
+					free(fullpath);
+					return NULL;
+				}
+			}
+		}
 		free(fullpath);
 	}
 
@@ -1321,6 +1409,7 @@ static int btrfs_list_subvols_print_v2(int fd,
 				    enum btrfs_list_layout layout,
 				    int is_list_all,
 				    int absolute_path,
+				    int follow_mount,
 				    const char *path,
 				    const char *raw_prefix)
 {
@@ -1330,7 +1419,7 @@ static int btrfs_list_subvols_print_v2(int fd,
 		subvols = btrfs_list_deleted_subvols(fd, filter_set);
 	else
 		subvols = btrfs_list_subvols(fd, is_list_all, absolute_path,
-					     path, filter_set);
+					     follow_mount, path, filter_set);
 	if (!subvols)
 		return -1;
 
@@ -1454,6 +1543,8 @@ static const char * const cmd_subvol_list_usage[] = {
 	"-a           print all the subvolumes in the filesystem and",
 	"             distinguish absolute and relative path with respect",
 	"             to the given <path> (require root privileges)",
+	"-f           follow mounted subvolumes below the specified path",
+	"             and list them too (only if it is the same filesystem)",
 	"",
 	"Field selection:",
 	"-p           print parent ID",
@@ -1497,6 +1588,7 @@ static int cmd_subvol_list(int argc, char **argv)
 	int ret = -1, uerr = 0;
 	char *subvol;
 	int is_list_all = 0;
+	int follow_mount = 0;
 	int is_only_in_path = 0;
 	int absolute_path = 0;
 	DIR *dirstream = NULL;
@@ -1513,7 +1605,7 @@ static int cmd_subvol_list(int argc, char **argv)
 		};
 
 		c = getopt_long(argc, argv,
-				    "acdgopqsurARG:C:t", long_options, NULL);
+				    "acdfgopqsurARG:C:t", long_options, NULL);
 		if (c < 0)
 			break;
 
@@ -1527,6 +1619,9 @@ static int cmd_subvol_list(int argc, char **argv)
 		case 'a':
 			is_list_all = 1;
 			break;
+		case 'f':
+			follow_mount = 1;
+			break;
 		case 'c':
 			btrfs_list_setup_print_column_v2(BTRFS_LIST_OGENERATION);
 			break;
@@ -1616,6 +1711,12 @@ static int cmd_subvol_list(int argc, char **argv)
 		goto out;
 	}
 
+	if (follow_mount && (is_list_all || is_only_in_path)) {
+		ret = -1;
+		error("cannot use -f with -a or -o option");
+		goto out;
+	}
+
 	subvol = argv[optind];
 	fd = btrfs_open_dir(subvol, &dirstream, 1);
 	if (fd < 0) {
@@ -1648,7 +1749,8 @@ static int cmd_subvol_list(int argc, char **argv)
 	btrfs_list_setup_print_column_v2(BTRFS_LIST_PATH);
 
 	ret = btrfs_list_subvols_print_v2(fd, filter_set, comparer_set,
-			layout, is_list_all, absolute_path, subvol, NULL);
+			layout, is_list_all, absolute_path, follow_mount,
+			subvol, NULL);
 
 out:
 	close_file_or_dir(fd, dirstream);
-- 
2.14.4

[PATCH v2 14/20] btrfs-progs: sub list: Update help message of -o option

From: Misono Tomohiro <hidden>
Date: 2018-06-18 08:38:18

Currently "sub list -o" lists only child subvolumes of the specified
path. So, update help message and variable name more appropriately.

Signed-off-by: Misono Tomohiro <redacted>
---
 Documentation/btrfs-subvolume.asciidoc |  2 +-
 cmds-subvolume.c                       | 10 +++++-----
 2 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/Documentation/btrfs-subvolume.asciidoc b/Documentation/btrfs-subvolume.asciidoc
index 20fae1e1..0381c92c 100644
--- a/Documentation/btrfs-subvolume.asciidoc
+++ b/Documentation/btrfs-subvolume.asciidoc
@@ -116,7 +116,7 @@ or at mount time via the subvolid= option.
 +
 Path filtering;;
 -o::::
-print only subvolumes below specified <path>.
+print only subvolumes which the subvolume of <path> contains.
 -a::::
 print all the subvolumes in the filesystem, including subvolumes
 which cannot be accessed from current mount point.
diff --git a/cmds-subvolume.c b/cmds-subvolume.c
index dab266aa..552c6dea 100644
--- a/cmds-subvolume.c
+++ b/cmds-subvolume.c
@@ -1550,7 +1550,7 @@ static const char * const cmd_subvol_list_usage[] = {
 	"It is possible to specify non-subvolume directory as <path>.",
 	"",
 	"Path filtering:",
-	"-o           print only subvolumes below specified path",
+	"-o           print only subvolumes which the subvolume of <path> contains",
 	"-a           print all the subvolumes in the filesystem.",
 	"             path to be shown is relative to the top-level",
 	"             subvolume (require root privileges)",
@@ -1605,7 +1605,7 @@ static int cmd_subvol_list(int argc, char **argv)
 	int follow_mount = 0;
 	int sort = 0;
 	int no_sort = 0;
-	int is_only_in_path = 0;
+	int is_only_child = 0;
 	int absolute_path = 0;
 	DIR *dirstream = NULL;
 	enum btrfs_list_layout layout = BTRFS_LIST_LAYOUT_DEFAULT;
@@ -1651,7 +1651,7 @@ static int cmd_subvol_list(int argc, char **argv)
 			btrfs_list_setup_print_column_v2(BTRFS_LIST_GENERATION);
 			break;
 		case 'o':
-			is_only_in_path = 1;
+			is_only_child = 1;
 			break;
 		case 't':
 			layout = BTRFS_LIST_LAYOUT_TABLE;
@@ -1732,7 +1732,7 @@ static int cmd_subvol_list(int argc, char **argv)
 		goto out;
 	}
 
-	if (follow_mount && (is_list_all || is_only_in_path)) {
+	if (follow_mount && (is_list_all || is_only_child)) {
 		ret = -1;
 		error("cannot use -f with -a or -o option");
 		goto out;
@@ -1760,7 +1760,7 @@ static int cmd_subvol_list(int argc, char **argv)
 	if (ret)
 		goto out;
 
-	if (is_only_in_path)
+	if (is_only_child)
 		btrfs_list_setup_filter_v2(&filter_set,
 					BTRFS_LIST_FILTER_TOPID_EQUAL,
 					top_id);
-- 
2.14.4

[PATCH v2 15/20] btrfs-progs: sub list: Update help message of -d option

From: Misono Tomohiro <hidden>
Date: 2018-06-18 08:38:18

Explicitly states that -d requires root privileges.
Also, update some option handling with regard to -d option.

Signed-off-by: Misono Tomohiro <redacted>
---
 Documentation/btrfs-subvolume.asciidoc | 3 ++-
 cmds-subvolume.c                       | 8 ++++++++
 2 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/Documentation/btrfs-subvolume.asciidoc b/Documentation/btrfs-subvolume.asciidoc
index 0381c92c..2db1d479 100644
--- a/Documentation/btrfs-subvolume.asciidoc
+++ b/Documentation/btrfs-subvolume.asciidoc
@@ -149,7 +149,8 @@ only snapshot subvolumes in the filesystem will be listed.
 -r::::
 only readonly subvolumes in the filesystem will be listed.
 -d::::
-list deleted subvolumes that are not yet cleaned.
+list deleted subvolumes that are not yet cleaned
+(require root privileges).
 
 Other;;
 -t::::
diff --git a/cmds-subvolume.c b/cmds-subvolume.c
index 552c6dea..ef39789a 100644
--- a/cmds-subvolume.c
+++ b/cmds-subvolume.c
@@ -1569,6 +1569,7 @@ static const char * const cmd_subvol_list_usage[] = {
 	"-s           list only snapshots",
 	"-r           list readonly subvolumes (including snapshots)",
 	"-d           list deleted subvolumes that are not yet cleaned",
+	"             (require root privileges)",
 	"",
 	"Other:",
 	"-t           print the result as a table",
@@ -1744,6 +1745,13 @@ static int cmd_subvol_list(int argc, char **argv)
 		goto out;
 	}
 
+	if (filter_set->only_deleted &&
+			(is_list_all || absolute_path || follow_mount)) {
+		ret = -1;
+		error("cannot use -d with -a/f/A option");
+		goto out;
+	}
+
 	subvol = argv[optind];
 	fd = btrfs_open_dir(subvol, &dirstream, 1);
 	if (fd < 0) {
-- 
2.14.4

[PATCH v2 16/20] btrfs-progs: utils: Fallback to open without O_NOATIME flag in find_mount_root():

From: Misono Tomohiro <hidden>
Date: 2018-06-18 08:38:18

O_NOATIME flag requires effective UID of process matches file's owner
or has CAP_FOWNER capabilities. Fallback to open without O_NOATIME flag
so that non-privileged user can also call find_mount_root().

This is a preparation work to allow non-privileged user to call
"subvolume show".

Signed-off-by: Misono Tomohiro <redacted>
---
 utils.c | 3 +++
 1 file changed, 3 insertions(+)
diff --git a/utils.c b/utils.c
index 21de09d3..1a4366e8 100644
--- a/utils.c
+++ b/utils.c
@@ -2048,6 +2048,9 @@ int find_mount_root(const char *path, char **mount_root)
 	char *longest_match = NULL;
 
 	fd = open(path, O_RDONLY | O_NOATIME);
+	if (fd < 0 && errno == EPERM)
+		fd = open(path, O_RDONLY);
+
 	if (fd < 0)
 		return -errno;
 	close(fd);
-- 
2.14.4

[PATCH v2 06/20] btrfs-progs: libbtrfsutil: Relax the privileges of subvolume iterator

From: Misono Tomohiro <hidden>
Date: 2018-06-18 08:38:19

This commit relax the privileges of subvolume iterator when the kernel
supports new ioctls (BTRFS_IOC_GET_SUBVOL_ROOTREF and
BTRFS_IOC_INO_LOOKUP_USER).

For older kernel (< 4.18), the behavior is the same.

Non-privileged user can use subvolume iterator only if new ioctls are
supported and btrfs_util_create_subvolume_iterator() is called with @top
zero (i.e. if the iterator is created from given path/fd). In this case:
 - an iterator can be created from non-subvolume directory
 - subvolume will be skipped if
   - it does not exist nor has different id from the found subvolume
     by INO_LOOKUP_USER (may happen if a dir in the path is being mounted)
   - it cannot be opened due to permission error

Note that this commit also allows root user to specify non-subvolume
direcotry when @top is zero. Only in that case, root's code path is
the same as user.

Signed-off-by: Misono Tomohiro <redacted>
---
 libbtrfsutil/btrfsutil.h |  18 ++-
 libbtrfsutil/errors.c    |   6 +
 libbtrfsutil/subvolume.c | 380 +++++++++++++++++++++++++++++++++++++++++++----
 3 files changed, 377 insertions(+), 27 deletions(-)
diff --git a/libbtrfsutil/btrfsutil.h b/libbtrfsutil/btrfsutil.h
index 3f21d163..49454a63 100644
--- a/libbtrfsutil/btrfsutil.h
+++ b/libbtrfsutil/btrfsutil.h
@@ -65,6 +65,10 @@ enum btrfs_util_error {
 	BTRFS_UTIL_ERROR_WAIT_SYNC_FAILED,
 	BTRFS_UTIL_ERROR_INVALID_ARGUMENT_FOR_USER,
 	BTRFS_UTIL_ERROR_GET_SUBVOL_INFO_FAILED,
+	BTRFS_UTIL_ERROR_GET_SUBVOL_ROOTREF_FAILED,
+	BTRFS_UTIL_ERROR_INO_LOOKUP_USER_FAILED,
+	BTRFS_UTIL_ERROR_DUP_FAILED,
+	BTRFS_UTIL_ERROR_CHDIR_FAILED,
 };
 
 /**
@@ -510,6 +514,14 @@ struct btrfs_util_subvolume_iterator;
  * @flags: Bitmask of BTRFS_UTIL_SUBVOLUME_ITERATOR_* flags.
  * @ret: Returned iterator.
  *
+ * Using subvolume iterator requires appropriate privilege (CAP_SYS_ADMIN) for
+ * kernel < 4.18. From kenrel >= 4.18 which supports
+ * BTRFS_IOC_GET_SUBVOL_ROOTREF and BTRFS_IOC_INO_LOOKUP_USER, non-previleged
+ * user can use it too (in that case @top must be zero). Also from kernel
+ * >=4.18, if @top is zero, the specified path can be non-subvolume directory
+ * and subvolumes which cannot be accessed will be skipped (either due to
+ * permission error or path is hidden by other mount).
+ *
  * The returned iterator must be freed with
  * btrfs_util_destroy_subvolume_iterator().
  *
@@ -558,7 +570,8 @@ int btrfs_util_subvolume_iterator_fd(const struct btrfs_util_subvolume_iterator
  * Must be freed with free().
  * @id_ret: Returned subvolume ID. May be %NULL.
  *
- * This requires appropriate privilege (CAP_SYS_ADMIN).
+ * This requires appropriate privilege (CAP_SYS_ADMIN) for kernel < 4.18.
+ * See the comment of btrfs_util_create_subvolume_iterator()
  *
  * Return: %BTRFS_UTIL_OK on success, %BTRFS_UTIL_ERROR_STOP_ITERATION if there
  * are no more subvolumes, non-zero error code on failure.
@@ -577,7 +590,8 @@ enum btrfs_util_error btrfs_util_subvolume_iterator_next(struct btrfs_util_subvo
  * This convenience function basically combines
  * btrfs_util_subvolume_iterator_next() and btrfs_util_subvolume_info().
  *
- * This requires appropriate privilege (CAP_SYS_ADMIN).
+ * This requires appropriate privilege (CAP_SYS_ADMIN) for kernel < 4.18.
+ * See the comment of btrfs_util_create_subvolume_iterator()
  *
  * Return: See btrfs_util_subvolume_iterator_next().
  */
diff --git a/libbtrfsutil/errors.c b/libbtrfsutil/errors.c
index f196fa71..c77407bf 100644
--- a/libbtrfsutil/errors.c
+++ b/libbtrfsutil/errors.c
@@ -49,6 +49,12 @@ static const char * const error_messages[] = {
 		"Non-root user cannot specify subvolume id",
 	[BTRFS_UTIL_ERROR_GET_SUBVOL_INFO_FAILED] =
 	"Could not get subvolume information by BTRFS_IOC_GET_SUBVOL_INFO",
+	[BTRFS_UTIL_ERROR_GET_SUBVOL_ROOTREF_FAILED] =
+	"Could not get rootref information by BTRRFS_IOC_GET_SUBVOL_ROOTREF",
+	[BTRFS_UTIL_ERROR_INO_LOOKUP_USER_FAILED] =
+	"Could not resolve subvolume path by BTRFS_IOC_INO_LOOKUP_USER",
+	[BTRFS_UTIL_ERROR_DUP_FAILED] = "Could not dup",
+	[BTRFS_UTIL_ERROR_CHDIR_FAILED] = "Could not chdir",
 };
 
 PUBLIC const char *btrfs_util_strerror(enum btrfs_util_error err)
diff --git a/libbtrfsutil/subvolume.c b/libbtrfsutil/subvolume.c
index 73471d4f..ae39d5c8 100644
--- a/libbtrfsutil/subvolume.c
+++ b/libbtrfsutil/subvolume.c
@@ -760,13 +760,21 @@ PUBLIC enum btrfs_util_error btrfs_util_create_subvolume_fd(int parent_fd,
 #define BTRFS_UTIL_SUBVOLUME_ITERATOR_CLOSE_FD (1 << 30)
 
 struct search_stack_entry {
+	/* below are used for subvolume_iterator_next_user */
+	uint64_t id;
+	struct btrfs_ioctl_get_subvol_rootref_args rootref_args;
+	/* below is used for subvolume_iterator_next_root */
 	struct btrfs_ioctl_search_args search;
+	/* below are used for both */
 	size_t items_pos, buf_off;
 	size_t path_len;
 };
 
 struct btrfs_util_subvolume_iterator {
+	bool use_tree_search;
 	int fd;
+	/* cur_fd is only used for subvolume_iterator_next_user */
+	int cur_fd;
 	int flags;
 
 	struct search_stack_entry *search_stack;
@@ -799,22 +807,28 @@ static enum btrfs_util_error append_to_search_stack(struct btrfs_util_subvolume_
 
 	entry = &iter->search_stack[iter->search_stack_len++];
 
-	memset(&entry->search, 0, sizeof(entry->search));
-	entry->search.key.tree_id = BTRFS_ROOT_TREE_OBJECTID;
-	entry->search.key.min_objectid = tree_id;
-	entry->search.key.max_objectid = tree_id;
-	entry->search.key.min_type = BTRFS_ROOT_REF_KEY;
-	entry->search.key.max_type = BTRFS_ROOT_REF_KEY;
-	entry->search.key.min_offset = 0;
-	entry->search.key.max_offset = UINT64_MAX;
-	entry->search.key.min_transid = 0;
-	entry->search.key.max_transid = UINT64_MAX;
-	entry->search.key.nr_items = 0;
-
-	entry->items_pos = 0;
-	entry->buf_off = 0;
-
-	entry->path_len = path_len;
+	if (iter->use_tree_search) {
+		memset(&entry->search, 0, sizeof(entry->search));
+		entry->search.key.tree_id = BTRFS_ROOT_TREE_OBJECTID;
+		entry->search.key.min_objectid = tree_id;
+		entry->search.key.max_objectid = tree_id;
+		entry->search.key.min_type = BTRFS_ROOT_REF_KEY;
+		entry->search.key.max_type = BTRFS_ROOT_REF_KEY;
+		entry->search.key.min_offset = 0;
+		entry->search.key.max_offset = UINT64_MAX;
+		entry->search.key.min_transid = 0;
+		entry->search.key.max_transid = UINT64_MAX;
+		entry->search.key.nr_items = 0;
+
+		entry->items_pos = 0;
+		entry->buf_off = 0;
+
+		entry->path_len = path_len;
+	} else {
+		memset(entry, 0, sizeof(*entry));
+		entry->path_len = path_len;
+		entry->id = tree_id;
+	}
 
 	return BTRFS_UTIL_OK;
 }
@@ -847,20 +861,32 @@ PUBLIC enum btrfs_util_error btrfs_util_create_subvolume_iterator_fd(int fd,
 {
 	struct btrfs_util_subvolume_iterator *iter;
 	enum btrfs_util_error err;
+	bool use_tree_search = true;
 
 	if (flags & ~BTRFS_UTIL_SUBVOLUME_ITERATOR_MASK) {
 		errno = EINVAL;
 		return BTRFS_UTIL_ERROR_INVALID_ARGUMENT;
 	}
 
-	if (top == 0) {
-		err = btrfs_util_is_subvolume_fd(fd);
-		if (err)
-			return err;
+	if (is_root()) {
+		if (top == 0) {
+			err = btrfs_util_is_subvolume_fd(fd);
+			if (err == BTRFS_UTIL_ERROR_NOT_SUBVOLUME) {
+				use_tree_search = false;
+			} else {
+				if (err)
+					return err;
 
-		err = btrfs_util_subvolume_id_fd(fd, &top);
-		if (err)
-			return err;
+				err = btrfs_util_subvolume_id_fd(fd, &top);
+				if (err)
+					return err;
+			}
+		}
+	} else {
+		if (top != 0)
+			return BTRFS_UTIL_ERROR_INVALID_ARGUMENT_FOR_USER;
+
+		use_tree_search = false;
 	}
 
 	iter = malloc(sizeof(*iter));
@@ -868,7 +894,15 @@ PUBLIC enum btrfs_util_error btrfs_util_create_subvolume_iterator_fd(int fd,
 		return BTRFS_UTIL_ERROR_NO_MEMORY;
 
 	iter->fd = fd;
+	if (!use_tree_search) {
+		iter->cur_fd = dup(fd);
+		if (iter->cur_fd < 0) {
+			err = BTRFS_UTIL_ERROR_DUP_FAILED;
+			goto out_iter;
+		}
+	}
 	iter->flags = flags;
+	iter->use_tree_search = use_tree_search;
 
 	iter->search_stack_len = 0;
 	iter->search_stack_capacity = 4;
@@ -1179,6 +1213,8 @@ PUBLIC void btrfs_util_destroy_subvolume_iterator(struct btrfs_util_subvolume_it
 		free(iter->search_stack);
 		if (iter->flags & BTRFS_UTIL_SUBVOLUME_ITERATOR_CLOSE_FD)
 			SAVE_ERRNO_AND_CLOSE(iter->fd);
+		if (iter->cur_fd)
+			SAVE_ERRNO_AND_CLOSE(iter->cur_fd);
 		free(iter);
 	}
 }
@@ -1255,6 +1291,74 @@ static enum btrfs_util_error build_subvol_path(struct btrfs_util_subvolume_itera
 	return BTRFS_UTIL_OK;
 }
 
+static enum btrfs_util_error build_subvol_path_user(struct btrfs_util_subvolume_iterator *iter,
+						    int fd,
+						    size_t *path_len_ret)
+{
+	struct search_stack_entry *top = top_search_stack_entry(iter);
+	struct btrfs_ioctl_ino_lookup_user_args args;
+	uint64_t dirid, treeid;
+	size_t dir_len, name_len, path_len;
+	char *p;
+	int ret;
+
+	dirid = top->rootref_args.rootref[top->items_pos].dirid;
+	treeid = top->rootref_args.rootref[top->items_pos].treeid;
+
+	memset(&args, 0, sizeof(args));
+	args.dirid = dirid;
+	args.treeid = treeid;
+
+	ret = ioctl(fd, BTRFS_IOC_INO_LOOKUP_USER, &args);
+	if (ret < 0)
+		return BTRFS_UTIL_ERROR_INO_LOOKUP_USER_FAILED;
+
+	dir_len = strlen(args.path);
+	name_len = strlen(args.name);
+	path_len = top->path_len;
+	/*
+	 * We need a joining slash if we have a current path and a subdirectory.
+	 */
+	if (top->path_len && dir_len)
+		path_len++;
+	path_len += dir_len;
+	/*
+	 * We need another joining slash if we have a current path and a name,
+	 * but not if we have a subdirectory, because the lookup ioctl includes
+	 * a trailing slash.
+	 */
+	if (top->path_len && !dir_len && name_len)
+		path_len++;
+	path_len += name_len;
+
+	if (path_len > iter->cur_path_capacity) {
+		/*
+		 * Allocate one more byte for NULL as we call open() in
+		 * subvolume_iterator_next_user()
+		 */
+		char *tmp = realloc(iter->cur_path, path_len + 1);
+
+		if (!tmp)
+			return BTRFS_UTIL_ERROR_NO_MEMORY;
+		iter->cur_path = tmp;
+		iter->cur_path_capacity = path_len;
+	}
+
+	p = iter->cur_path + top->path_len;
+	if (top->path_len && dir_len)
+		*p++ = '/';
+	memcpy(p, args.path, dir_len);
+	p += dir_len;
+	if (top->path_len && !dir_len && name_len)
+		*p++ = '/';
+	memcpy(p, args.name, name_len);
+	p += name_len;
+
+	*path_len_ret = path_len;
+
+	return BTRFS_UTIL_OK;
+}
+
 static enum btrfs_util_error subvolume_iterator_next_root(struct btrfs_util_subvolume_iterator *iter,
 							  char **path_ret,
 							  uint64_t *id_ret)
@@ -1331,11 +1435,234 @@ out:
 	return BTRFS_UTIL_OK;
 }
 
+static enum btrfs_util_error chdir_to_parent_subvolume(struct btrfs_util_subvolume_iterator *iter,
+						       struct search_stack_entry *top)
+{
+	size_t start, end;
+	int ret;
+
+	/* end indicates name length of current subvolume */
+	end = top->path_len;
+
+	/* start indicates name length of current's parent subvolume */
+	top = top_search_stack_entry(iter);
+	start = top->path_len;
+	if (start)
+		start++;
+
+	/* move up directory as many times '/' appear */
+	while (start < end) {
+		if (iter->cur_path[start] == '/') {
+			ret = chdir("..");
+			if (ret)
+				return BTRFS_UTIL_ERROR_CHDIR_FAILED;
+		}
+		start++;
+	}
+	ret = chdir("..");
+	if (ret)
+		return BTRFS_UTIL_ERROR_CHDIR_FAILED;
+
+	iter->cur_fd = open(".", O_RDONLY);
+	if (iter->cur_fd < 0)
+		return BTRFS_UTIL_ERROR_OPEN_FAILED;
+
+	return BTRFS_UTIL_OK;
+}
+
+static enum btrfs_util_error subvolume_iterator_next_user(struct btrfs_util_subvolume_iterator *iter,
+							  char **path_ret,
+							  uint64_t *id_ret)
+{
+	struct search_stack_entry *top;
+	size_t path_len;
+	enum btrfs_util_error err;
+	char *temp;
+	int child_fd, orig_fd;
+	int ret;
+
+	/* Remember current working directry to reset at return */
+	orig_fd = open(".", O_RDONLY);
+	if (orig_fd < 0)
+		return BTRFS_UTIL_ERROR_OPEN_FAILED;
+
+	/*
+	 * Change working directory as traversing iterator,
+	 * in order to prevent ENAMETOOLONG error at open()
+	 */
+	ret = fchdir(iter->cur_fd);
+	if (ret) {
+		SAVE_ERRNO_AND_CLOSE(orig_fd);
+		return BTRFS_UTIL_ERROR_CHDIR_FAILED;
+	}
+	if (iter->flags & BTRFS_UTIL_SUBVOLUME_ITERATOR_POST_ORDER &&
+	    iter->search_stack_len > 1) {
+		SAVE_ERRNO_AND_CLOSE(iter->cur_fd);
+		top = top_search_stack_entry(iter);
+		iter->search_stack_len--;
+
+		err = chdir_to_parent_subvolume(iter, top);
+		if (err) {
+			SAVE_ERRNO_AND_CLOSE(orig_fd);
+			return err;
+		}
+	}
+
+	for (;;) {
+		for (;;) {
+			if (iter->search_stack_len == 0) {
+				ret = fchdir(orig_fd);
+				SAVE_ERRNO_AND_CLOSE(orig_fd);
+				if (ret)
+					return BTRFS_UTIL_ERROR_CHDIR_FAILED;
+				return BTRFS_UTIL_ERROR_STOP_ITERATION;
+			}
+
+			top = top_search_stack_entry(iter);
+
+			if (top->items_pos < top->rootref_args.num_items) {
+				break;
+			} else {
+				top->items_pos = 0;
+				ret = ioctl(iter->cur_fd, BTRFS_IOC_GET_SUBVOL_ROOTREF,
+						&top->rootref_args);
+				if (ret < 0) {
+					if (errno != EOVERFLOW) {
+						SAVE_ERRNO_AND_CLOSE(orig_fd);
+						return BTRFS_UTIL_ERROR_GET_SUBVOL_ROOTREF_FAILED;
+					}
+				} else {
+					if (top->rootref_args.num_items == 0) {
+						if (iter->flags & BTRFS_UTIL_SUBVOLUME_ITERATOR_POST_ORDER) {
+							if (iter->search_stack_len == 1) {
+								ret = fchdir(orig_fd);
+								SAVE_ERRNO_AND_CLOSE(orig_fd);
+								if (ret)
+									return BTRFS_UTIL_ERROR_CHDIR_FAILED;
+								return BTRFS_UTIL_ERROR_STOP_ITERATION;
+							}
+							goto out;
+						}
+
+						iter->search_stack_len--;
+						SAVE_ERRNO_AND_CLOSE(iter->cur_fd);
+
+						err = chdir_to_parent_subvolume(iter, top);
+						if (err) {
+							SAVE_ERRNO_AND_CLOSE(orig_fd);
+							return err;
+						}
+					}
+				}
+			}
+		}
+
+		err = build_subvol_path_user(iter, iter->cur_fd,
+				&path_len);
+		top->id = top->rootref_args.rootref[top->items_pos].treeid;
+		top->items_pos++;
+
+		/* Skip if permission error happens during path construction */
+		if (err) {
+			if (errno == EACCES)
+				continue;
+			SAVE_ERRNO_AND_CLOSE(orig_fd);
+			return err;
+		}
+
+		/* Extra check to the constructed path */
+		iter->cur_path[path_len] = '\0';
+		temp = &iter->cur_path[top->path_len];
+		if (*temp == '/')
+			temp++;
+		child_fd = open(temp, O_RDONLY);
+		if (child_fd < 0) {
+			/*
+			 * Skip if
+			 *  1. the path does not exist
+			 *     (may happen if a dir in the path is mounted)
+			 *  2. cannot open the path due to permission error
+			 */
+			if (errno == ENOENT || errno == EACCES)
+				continue;
+			SAVE_ERRNO_AND_CLOSE(orig_fd);
+			return BTRFS_UTIL_ERROR_OPEN_FAILED;
+		} else {
+			uint64_t temp_id;
+
+			/*
+			 * Check the subvolume status and skip if other volume
+			 * is mounted
+			 */
+			err = btrfs_util_is_subvolume_fd(child_fd);
+			if (err) {
+				SAVE_ERRNO_AND_CLOSE(child_fd);
+				if (err == BTRFS_UTIL_ERROR_NOT_BTRFS ||
+				    err == BTRFS_UTIL_ERROR_NOT_SUBVOLUME)
+					continue;
+				SAVE_ERRNO_AND_CLOSE(orig_fd);
+				return err;
+			}
+			err = btrfs_util_subvolume_id_fd(child_fd, &temp_id);
+			if (err) {
+				SAVE_ERRNO_AND_CLOSE(orig_fd);
+				SAVE_ERRNO_AND_CLOSE(child_fd);
+				return err;
+			}
+			if (top->id != temp_id) {
+				SAVE_ERRNO_AND_CLOSE(child_fd);
+				continue;
+			}
+		}
+
+		err = append_to_search_stack(iter, top->id, path_len);
+		if (err) {
+			SAVE_ERRNO_AND_CLOSE(orig_fd);
+			SAVE_ERRNO_AND_CLOSE(child_fd);
+			return err;
+		}
+
+		SAVE_ERRNO_AND_CLOSE(iter->cur_fd);
+		iter->cur_fd = child_fd;
+
+		if (!(iter->flags & BTRFS_UTIL_SUBVOLUME_ITERATOR_POST_ORDER)) {
+			top = top_search_stack_entry(iter);
+			goto out;
+		} else {
+			ret = fchdir(iter->cur_fd);
+			if (ret) {
+				SAVE_ERRNO_AND_CLOSE(orig_fd);
+				return BTRFS_UTIL_ERROR_CHDIR_FAILED;
+			}
+		}
+	}
+
+out:
+	ret = fchdir(orig_fd);
+	SAVE_ERRNO_AND_CLOSE(orig_fd);
+	if (ret)
+		return BTRFS_UTIL_ERROR_CHDIR_FAILED;
+
+	if (path_ret) {
+		*path_ret = malloc(top->path_len + 1);
+		if (!*path_ret)
+			return BTRFS_UTIL_ERROR_NO_MEMORY;
+		memcpy(*path_ret, iter->cur_path, top->path_len);
+		(*path_ret)[top->path_len] = '\0';
+	}
+	if (id_ret)
+		*id_ret = top->id;
+	return BTRFS_UTIL_OK;
+}
+
 PUBLIC enum btrfs_util_error btrfs_util_subvolume_iterator_next(struct btrfs_util_subvolume_iterator *iter,
 								char **path_ret,
 								uint64_t *id_ret)
 {
-	return subvolume_iterator_next_root(iter, path_ret, id_ret);
+	if (iter->use_tree_search)
+		return subvolume_iterator_next_root(iter, path_ret, id_ret);
+	else
+		return subvolume_iterator_next_user(iter, path_ret, id_ret);
 }
 
 PUBLIC enum btrfs_util_error btrfs_util_subvolume_iterator_next_info(struct btrfs_util_subvolume_iterator *iter,
@@ -1349,7 +1676,10 @@ PUBLIC enum btrfs_util_error btrfs_util_subvolume_iterator_next_info(struct btrf
 	if (err)
 		return err;
 
-	return btrfs_util_subvolume_info_fd(iter->fd, id, subvol);
+	if (iter->use_tree_search)
+		return btrfs_util_subvolume_info_fd(iter->fd, id, subvol);
+	else
+		return btrfs_util_subvolume_info_fd(iter->cur_fd, 0, subvol);
 }
 
 PUBLIC enum btrfs_util_error btrfs_util_deleted_subvolumes(const char *path,
-- 
2.14.4

[PATCH v2 07/20] btrfs-progs: sub list: Use libbtrfsuitl for subvolume list

From: Misono Tomohiro <hidden>
Date: 2018-06-18 08:38:19

This is a copy of non-merged following patch originally written
by Omar Sandoval:
  btrfs-progs: use libbtrfsutil for subvolume list
expect this commit keeps libbtrfs implementation which above commit
tries to remove (therefore this adds suffix _v2 for struct/function).

Original Author: Omar Sandoval [off-list ref]
Signed-off-by: Misono Tomohiro <redacted>
---
 cmds-subvolume.c | 963 +++++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 936 insertions(+), 27 deletions(-)
diff --git a/cmds-subvolume.c b/cmds-subvolume.c
index 45363a5a..c54a8003 100644
--- a/cmds-subvolume.c
+++ b/cmds-subvolume.c
@@ -404,6 +404,915 @@ keep_fd:
 	return ret;
 }
 
+#define BTRFS_LIST_NFILTERS_INCREASE	(2 * BTRFS_LIST_FILTER_MAX)
+#define BTRFS_LIST_NCOMPS_INCREASE	(2 * BTRFS_LIST_COMP_MAX)
+
+struct listed_subvol {
+	struct btrfs_util_subvolume_info info;
+	char *path;
+};
+
+struct subvol_list {
+	size_t num;
+	struct listed_subvol subvols[];
+};
+
+typedef int (*btrfs_list_filter_func_v2)(struct listed_subvol *, uint64_t);
+typedef int (*btrfs_list_comp_func_v2)(const struct listed_subvol *,
+				    const struct listed_subvol *,
+				    int);
+
+struct btrfs_list_filter_v2 {
+	btrfs_list_filter_func_v2 filter_func;
+	u64 data;
+};
+
+struct btrfs_list_comparer_v2 {
+	btrfs_list_comp_func_v2 comp_func;
+	int is_descending;
+};
+
+struct btrfs_list_filter_set_v2 {
+	int total;
+	int nfilters;
+	int only_deleted;
+	struct btrfs_list_filter_v2 filters[0];
+};
+
+struct btrfs_list_comparer_set_v2 {
+	int total;
+	int ncomps;
+	struct btrfs_list_comparer_v2 comps[0];
+};
+
+static struct {
+	char	*name;
+	char	*column_name;
+	int	need_print;
+} btrfs_list_columns[] = {
+	{
+		.name		= "ID",
+		.column_name	= "ID",
+		.need_print	= 0,
+	},
+	{
+		.name		= "gen",
+		.column_name	= "Gen",
+		.need_print	= 0,
+	},
+	{
+		.name		= "cgen",
+		.column_name	= "CGen",
+		.need_print	= 0,
+	},
+	{
+		.name		= "parent",
+		.column_name	= "Parent",
+		.need_print	= 0,
+	},
+	{
+		.name		= "top level",
+		.column_name	= "Top Level",
+		.need_print	= 0,
+	},
+	{
+		.name		= "otime",
+		.column_name	= "OTime",
+		.need_print	= 0,
+	},
+	{
+		.name		= "parent_uuid",
+		.column_name	= "Parent UUID",
+		.need_print	= 0,
+	},
+	{
+		.name		= "received_uuid",
+		.column_name	= "Received UUID",
+		.need_print	= 0,
+	},
+	{
+		.name		= "uuid",
+		.column_name	= "UUID",
+		.need_print	= 0,
+	},
+	{
+		.name		= "path",
+		.column_name	= "Path",
+		.need_print	= 0,
+	},
+	{
+		.name		= NULL,
+		.column_name	= NULL,
+		.need_print	= 0,
+	},
+};
+
+static btrfs_list_filter_func_v2 all_filter_funcs[];
+static btrfs_list_comp_func_v2 all_comp_funcs[];
+
+static void btrfs_list_setup_print_column_v2(enum btrfs_list_column_enum column)
+{
+	int i;
+
+	ASSERT(0 <= column && column <= BTRFS_LIST_ALL);
+
+	if (column < BTRFS_LIST_ALL) {
+		btrfs_list_columns[column].need_print = 1;
+		return;
+	}
+
+	for (i = 0; i < BTRFS_LIST_ALL; i++)
+		btrfs_list_columns[i].need_print = 1;
+}
+
+static int comp_entry_with_rootid_v2(const struct listed_subvol *entry1,
+				  const struct listed_subvol *entry2,
+				  int is_descending)
+{
+	int ret;
+
+	if (entry1->info.id > entry2->info.id)
+		ret = 1;
+	else if (entry1->info.id < entry2->info.id)
+		ret = -1;
+	else
+		ret = 0;
+
+	return is_descending ? -ret : ret;
+}
+
+static int comp_entry_with_gen_v2(const struct listed_subvol *entry1,
+			       const struct listed_subvol *entry2,
+			       int is_descending)
+{
+	int ret;
+
+	if (entry1->info.generation > entry2->info.generation)
+		ret = 1;
+	else if (entry1->info.generation < entry2->info.generation)
+		ret = -1;
+	else
+		ret = 0;
+
+	return is_descending ? -ret : ret;
+}
+
+static int comp_entry_with_ogen_v2(const struct listed_subvol *entry1,
+				const struct listed_subvol *entry2,
+				int is_descending)
+{
+	int ret;
+
+	if (entry1->info.otransid > entry2->info.otransid)
+		ret = 1;
+	else if (entry1->info.otransid < entry2->info.otransid)
+		ret = -1;
+	else
+		ret = 0;
+
+	return is_descending ? -ret : ret;
+}
+
+static int comp_entry_with_path_v2(const struct listed_subvol *entry1,
+				const struct listed_subvol *entry2,
+				int is_descending)
+{
+	int ret;
+
+	if (strcmp(entry1->path, entry2->path) > 0)
+		ret = 1;
+	else if (strcmp(entry1->path, entry2->path) < 0)
+		ret = -1;
+	else
+		ret = 0;
+
+	return is_descending ? -ret : ret;
+}
+
+static btrfs_list_comp_func_v2 all_comp_funcs[] = {
+	[BTRFS_LIST_COMP_ROOTID]	= comp_entry_with_rootid_v2,
+	[BTRFS_LIST_COMP_OGEN]		= comp_entry_with_ogen_v2,
+	[BTRFS_LIST_COMP_GEN]		= comp_entry_with_gen_v2,
+	[BTRFS_LIST_COMP_PATH]		= comp_entry_with_path_v2,
+};
+
+static char *all_sort_items[] = {
+	[BTRFS_LIST_COMP_ROOTID]	= "rootid",
+	[BTRFS_LIST_COMP_OGEN]		= "ogen",
+	[BTRFS_LIST_COMP_GEN]		= "gen",
+	[BTRFS_LIST_COMP_PATH]		= "path",
+	[BTRFS_LIST_COMP_MAX]		= NULL,
+};
+
+static int btrfs_list_get_sort_item(char *sort_name)
+{
+	int i;
+
+	for (i = 0; i < BTRFS_LIST_COMP_MAX; i++) {
+		if (strcmp(sort_name, all_sort_items[i]) == 0)
+			return i;
+	}
+	return -1;
+}
+
+static struct btrfs_list_comparer_set_v2 *btrfs_list_alloc_comparer_set_v2(void)
+{
+	struct btrfs_list_comparer_set_v2 *set;
+	int size;
+
+	size = sizeof(struct btrfs_list_comparer_set_v2) +
+		BTRFS_LIST_NCOMPS_INCREASE *
+		sizeof(struct btrfs_list_comparer_v2);
+	set = calloc(1, size);
+	if (!set) {
+		fprintf(stderr, "memory allocation failed\n");
+		exit(1);
+	}
+
+	set->total = BTRFS_LIST_NCOMPS_INCREASE;
+
+	return set;
+}
+
+static int btrfs_list_setup_comparer_v2(struct btrfs_list_comparer_set_v2 **comp_set,
+				     enum btrfs_list_comp_enum comparer,
+				     int is_descending)
+{
+	struct btrfs_list_comparer_set_v2 *set = *comp_set;
+	int size;
+
+	ASSERT(set != NULL);
+	ASSERT(comparer < BTRFS_LIST_COMP_MAX);
+	ASSERT(set->ncomps <= set->total);
+
+	if (set->ncomps == set->total) {
+		void *tmp;
+
+		size = set->total + BTRFS_LIST_NCOMPS_INCREASE;
+		size = sizeof(*set) +
+			size * sizeof(struct btrfs_list_comparer_v2);
+		tmp = set;
+		set = realloc(set, size);
+		if (!set) {
+			fprintf(stderr, "memory allocation failed\n");
+			free(tmp);
+			exit(1);
+		}
+
+		memset(&set->comps[set->total], 0,
+		       BTRFS_LIST_NCOMPS_INCREASE *
+		       sizeof(struct btrfs_list_comparer_v2));
+		set->total += BTRFS_LIST_NCOMPS_INCREASE;
+		*comp_set = set;
+	}
+
+	ASSERT(set->comps[set->ncomps].comp_func == NULL);
+
+	set->comps[set->ncomps].comp_func = all_comp_funcs[comparer];
+	set->comps[set->ncomps].is_descending = is_descending;
+	set->ncomps++;
+	return 0;
+}
+
+static int subvol_comp(const void *entry1, const void *entry2, void *arg)
+{
+	const struct btrfs_list_comparer_set_v2 * const set = arg;
+	int rootid_compared = 0;
+	int ret;
+	int i;
+
+	for (i = 0; set && i < set->ncomps; i++) {
+		if (!set->comps[i].comp_func)
+			break;
+
+		ret = set->comps[i].comp_func(entry1, entry2,
+					      set->comps[i].is_descending);
+		if (ret)
+			return ret;
+
+		if (set->comps[i].comp_func == comp_entry_with_rootid_v2)
+			rootid_compared = 1;
+	}
+
+	if (!rootid_compared)
+		return comp_entry_with_rootid_v2(entry1, entry2, 0);
+
+	return 0;
+}
+
+static void sort_subvols(struct btrfs_list_comparer_set_v2 *comp_set,
+			 struct subvol_list *subvols)
+{
+	qsort_r(subvols->subvols, subvols->num, sizeof(subvols->subvols[0]),
+		subvol_comp, comp_set);
+}
+
+static int filter_by_rootid(struct listed_subvol *subvol, uint64_t data)
+{
+	return subvol->info.id == data;
+}
+
+static int filter_snapshot(struct listed_subvol *subvol, uint64_t data)
+{
+	return !uuid_is_null(subvol->info.parent_uuid);
+}
+
+static int filter_flags(struct listed_subvol *subvol, uint64_t data)
+{
+	return subvol->info.flags & data;
+}
+
+static int filter_gen_more(struct listed_subvol *subvol, uint64_t data)
+{
+	return subvol->info.generation >= data;
+}
+
+static int filter_gen_less(struct listed_subvol *subvol, uint64_t data)
+{
+	return subvol->info.generation <= data;
+}
+
+static int filter_gen_equal(struct listed_subvol *subvol, uint64_t data)
+{
+	return subvol->info.generation == data;
+}
+
+static int filter_cgen_more(struct listed_subvol *subvol, uint64_t data)
+{
+	return subvol->info.otransid >= data;
+}
+
+static int filter_cgen_less(struct listed_subvol *subvol, uint64_t data)
+{
+	return subvol->info.otransid <= data;
+}
+
+static int filter_cgen_equal(struct listed_subvol *subvol, uint64_t data)
+{
+	return subvol->info.otransid == data;
+}
+
+static int filter_topid_equal(struct listed_subvol *subvol, uint64_t data)
+{
+	/* See the comment in print_subvolume_column() about top level. */
+	return subvol->info.parent_id == data;
+}
+
+static int filter_full_path(struct listed_subvol *subvol, uint64_t data)
+{
+	/*
+	 * This implements the same behavior as before the conversion to
+	 * libbtrfsutil, which is mostly nonsensical.
+	 */
+	if (subvol->info.parent_id != data) {
+		char *tmp;
+		int ret;
+
+		ret = asprintf(&tmp, "<FS_TREE>/%s", subvol->path);
+		if (ret == -1) {
+			error("out of memory");
+			exit(1);
+		}
+
+		free(subvol->path);
+		subvol->path = tmp;
+	}
+	return 1;
+}
+
+static int filter_by_parent(struct listed_subvol *subvol, uint64_t data)
+{
+	return !uuid_compare(subvol->info.parent_uuid,
+			     (u8 *)(unsigned long)data);
+}
+
+static btrfs_list_filter_func_v2 all_filter_funcs[] = {
+	[BTRFS_LIST_FILTER_ROOTID]		= filter_by_rootid,
+	[BTRFS_LIST_FILTER_SNAPSHOT_ONLY]	= filter_snapshot,
+	[BTRFS_LIST_FILTER_FLAGS]		= filter_flags,
+	[BTRFS_LIST_FILTER_GEN_MORE]		= filter_gen_more,
+	[BTRFS_LIST_FILTER_GEN_LESS]		= filter_gen_less,
+	[BTRFS_LIST_FILTER_GEN_EQUAL]           = filter_gen_equal,
+	[BTRFS_LIST_FILTER_CGEN_MORE]		= filter_cgen_more,
+	[BTRFS_LIST_FILTER_CGEN_LESS]		= filter_cgen_less,
+	[BTRFS_LIST_FILTER_CGEN_EQUAL]          = filter_cgen_equal,
+	[BTRFS_LIST_FILTER_TOPID_EQUAL]		= filter_topid_equal,
+	[BTRFS_LIST_FILTER_FULL_PATH]		= filter_full_path,
+	[BTRFS_LIST_FILTER_BY_PARENT]		= filter_by_parent,
+};
+
+static struct btrfs_list_filter_set_v2 *btrfs_list_alloc_filter_set_v2(void)
+{
+	struct btrfs_list_filter_set_v2 *set;
+	int size;
+
+	size = sizeof(struct btrfs_list_filter_set_v2) +
+		BTRFS_LIST_NFILTERS_INCREASE *
+		sizeof(struct btrfs_list_filter_v2);
+	set = calloc(1, size);
+	if (!set) {
+		fprintf(stderr, "memory allocation failed\n");
+		exit(1);
+	}
+
+	set->total = BTRFS_LIST_NFILTERS_INCREASE;
+
+	return set;
+}
+
+/*
+ * Setup list filters. Exit if there's not enough memory, as we can't continue
+ * without the structures set up properly.
+ */
+static void btrfs_list_setup_filter_v2(struct btrfs_list_filter_set_v2 **filter_set,
+				    enum btrfs_list_filter_enum filter,
+				    u64 data)
+{
+	struct btrfs_list_filter_set_v2 *set = *filter_set;
+	int size;
+
+	ASSERT(set != NULL);
+	ASSERT(filter < BTRFS_LIST_FILTER_MAX);
+	ASSERT(set->nfilters <= set->total);
+
+	if (set->nfilters == set->total) {
+		void *tmp;
+
+		size = set->total + BTRFS_LIST_NFILTERS_INCREASE;
+		size = sizeof(*set) +
+			size * sizeof(struct btrfs_list_filter_v2);
+		tmp = set;
+		set = realloc(set, size);
+		if (!set) {
+			fprintf(stderr, "memory allocation failed\n");
+			free(tmp);
+			exit(1);
+		}
+
+		memset(&set->filters[set->total], 0,
+		       BTRFS_LIST_NFILTERS_INCREASE *
+		       sizeof(struct btrfs_list_filter_v2));
+		set->total += BTRFS_LIST_NFILTERS_INCREASE;
+		*filter_set = set;
+	}
+
+	ASSERT(set->filters[set->nfilters].filter_func == NULL);
+
+	if (filter == BTRFS_LIST_FILTER_DELETED) {
+		set->only_deleted = 1;
+	} else {
+		set->filters[set->nfilters].filter_func =
+					all_filter_funcs[filter];
+		set->filters[set->nfilters].data = data;
+		set->nfilters++;
+	}
+}
+
+static int filters_match(struct listed_subvol *subvol,
+			 struct btrfs_list_filter_set_v2 *set)
+{
+	int i, ret;
+
+	if (!set)
+		return 1;
+
+	for (i = 0; i < set->nfilters; i++) {
+		if (!set->filters[i].filter_func)
+			break;
+		ret = set->filters[i].filter_func(subvol, set->filters[i].data);
+		if (!ret)
+			return 0;
+	}
+	return 1;
+}
+
+static void print_subvolume_column(struct listed_subvol *subvol,
+				   enum btrfs_list_column_enum column)
+{
+	char tstr[256];
+	char uuidparse[BTRFS_UUID_UNPARSED_SIZE];
+
+	ASSERT(0 <= column && column < BTRFS_LIST_ALL);
+
+	switch (column) {
+	case BTRFS_LIST_OBJECTID:
+		printf("%" PRIu64, subvol->info.id);
+		break;
+	case BTRFS_LIST_GENERATION:
+		printf("%" PRIu64, subvol->info.generation);
+		break;
+	case BTRFS_LIST_OGENERATION:
+		printf("%" PRIu64, subvol->info.otransid);
+		break;
+	case BTRFS_LIST_PARENT:
+	/*
+	 * Top level used to mean something else, but since 4f5ebb3ef553
+	 * ("Btrfs-progs: fix to make list specified directory's subvolumes
+	 * work") it was always set to the parent ID. See
+	 * https://www.spinics.net/lists/linux-btrfs/msg69820.html.
+	 */
+	case BTRFS_LIST_TOP_LEVEL:
+		printf("%" PRIu64, subvol->info.parent_id);
+		break;
+	case BTRFS_LIST_OTIME:
+		if (subvol->info.otime.tv_sec) {
+			struct tm tm;
+
+			localtime_r(&subvol->info.otime.tv_sec, &tm);
+			strftime(tstr, sizeof(tstr), "%Y-%m-%d %X", &tm);
+		} else
+			strcpy(tstr, "-");
+		printf("%s", tstr);
+		break;
+	case BTRFS_LIST_UUID:
+		if (uuid_is_null(subvol->info.uuid))
+			strcpy(uuidparse, "-");
+		else
+			uuid_unparse(subvol->info.uuid, uuidparse);
+		printf("%-36s", uuidparse);
+		break;
+	case BTRFS_LIST_PUUID:
+		if (uuid_is_null(subvol->info.parent_uuid))
+			strcpy(uuidparse, "-");
+		else
+			uuid_unparse(subvol->info.parent_uuid, uuidparse);
+		printf("%-36s", uuidparse);
+		break;
+	case BTRFS_LIST_RUUID:
+		if (uuid_is_null(subvol->info.received_uuid))
+			strcpy(uuidparse, "-");
+		else
+			uuid_unparse(subvol->info.received_uuid, uuidparse);
+		printf("%-36s", uuidparse);
+		break;
+	case BTRFS_LIST_PATH:
+		printf("%s", subvol->path);
+		break;
+	default:
+		break;
+	}
+}
+
+static void print_one_subvol_info_raw(struct listed_subvol *subvol,
+				      const char *raw_prefix)
+{
+	int i;
+
+	for (i = 0; i < BTRFS_LIST_ALL; i++) {
+		if (!btrfs_list_columns[i].need_print)
+			continue;
+
+		if (raw_prefix)
+			printf("%s", raw_prefix);
+
+		print_subvolume_column(subvol, i);
+	}
+	printf("\n");
+}
+
+static void print_one_subvol_info_table(struct listed_subvol *subvol)
+{
+	int i;
+
+	for (i = 0; i < BTRFS_LIST_ALL; i++) {
+		if (!btrfs_list_columns[i].need_print)
+			continue;
+
+		print_subvolume_column(subvol, i);
+
+		if (i != BTRFS_LIST_PATH)
+			printf("\t");
+
+		if (i == BTRFS_LIST_TOP_LEVEL)
+			printf("\t");
+	}
+	printf("\n");
+}
+
+static void print_one_subvol_info_default(struct listed_subvol *subvol)
+{
+	int i;
+
+	for (i = 0; i < BTRFS_LIST_ALL; i++) {
+		if (!btrfs_list_columns[i].need_print)
+			continue;
+
+		printf("%s ", btrfs_list_columns[i].name);
+		print_subvolume_column(subvol, i);
+
+		if (i != BTRFS_LIST_PATH)
+			printf(" ");
+	}
+	printf("\n");
+}
+
+static void print_all_subvol_info_tab_head(void)
+{
+	int i;
+	int len;
+	char barrier[20];
+
+	for (i = 0; i < BTRFS_LIST_ALL; i++) {
+		if (btrfs_list_columns[i].need_print)
+			printf("%s\t", btrfs_list_columns[i].name);
+
+		if (i == BTRFS_LIST_ALL-1)
+			printf("\n");
+	}
+
+	for (i = 0; i < BTRFS_LIST_ALL; i++) {
+		memset(barrier, 0, sizeof(barrier));
+
+		if (btrfs_list_columns[i].need_print) {
+			len = strlen(btrfs_list_columns[i].name);
+			while (len--)
+				strcat(barrier, "-");
+
+			printf("%s\t", barrier);
+		}
+		if (i == BTRFS_LIST_ALL-1)
+			printf("\n");
+	}
+}
+
+static void print_all_subvol_info(struct subvol_list *subvols,
+				  enum btrfs_list_layout layout,
+				  const char *raw_prefix)
+{
+	size_t i;
+
+	if (layout == BTRFS_LIST_LAYOUT_TABLE)
+		print_all_subvol_info_tab_head();
+
+	for (i = 0; i < subvols->num; i++) {
+		struct listed_subvol *subvol = &subvols->subvols[i];
+
+		switch (layout) {
+		case BTRFS_LIST_LAYOUT_DEFAULT:
+			print_one_subvol_info_default(subvol);
+			break;
+		case BTRFS_LIST_LAYOUT_TABLE:
+			print_one_subvol_info_table(subvol);
+			break;
+		case BTRFS_LIST_LAYOUT_RAW:
+			print_one_subvol_info_raw(subvol, raw_prefix);
+			break;
+		}
+	}
+}
+
+static void free_subvol_list(struct subvol_list *subvols)
+{
+	size_t i;
+
+	if (subvols) {
+		for (i = 0; i < subvols->num; i++)
+			free(subvols->subvols[i].path);
+		free(subvols);
+	}
+}
+
+static struct subvol_list *btrfs_list_deleted_subvols(int fd,
+						      struct btrfs_list_filter_set_v2 *filter_set)
+{
+	struct subvol_list *subvols = NULL;
+	uint64_t *ids = NULL;
+	size_t i, n;
+	enum btrfs_util_error err;
+	int ret = -1;
+
+	err = btrfs_util_deleted_subvolumes_fd(fd, &ids, &n);
+	if (err) {
+		error_btrfs_util(err);
+		return NULL;
+	}
+
+	subvols = malloc(sizeof(*subvols) + n * sizeof(subvols->subvols[0]));
+	if (!subvols) {
+		error("out of memory");
+		goto out;
+	}
+
+	subvols->num = 0;
+	for (i = 0; i < n; i++) {
+		struct listed_subvol *subvol = &subvols->subvols[subvols->num];
+
+		err = btrfs_util_subvolume_info_fd(fd, ids[i], &subvol->info);
+		if (err) {
+			error_btrfs_util(err);
+			goto out;
+		}
+
+		subvol->path = strdup("DELETED");
+		if (!subvol->path)
+			goto out;
+
+		if (!filters_match(subvol, filter_set)) {
+			free(subvol->path);
+			continue;
+		}
+
+		subvols->num++;
+	}
+
+	ret = 0;
+out:
+	if (ret) {
+		free_subvol_list(subvols);
+		subvols = NULL;
+		free(ids);
+	}
+	return subvols;
+}
+
+static struct subvol_list *btrfs_list_subvols(int fd,
+					      struct btrfs_list_filter_set_v2 *filter_set)
+{
+	struct subvol_list *subvols;
+	size_t capacity = 0;
+	struct btrfs_util_subvolume_iterator *iter;
+	enum btrfs_util_error err;
+	int ret = -1;
+
+	subvols = malloc(sizeof(*subvols));
+	if (!subvols) {
+		error("out of memory");
+		return NULL;
+	}
+	subvols->num = 0;
+
+	err = btrfs_util_create_subvolume_iterator_fd(fd,
+						      BTRFS_FS_TREE_OBJECTID, 0,
+						      &iter);
+	if (err) {
+		iter = NULL;
+		error_btrfs_util(err);
+		goto out;
+	}
+
+	for (;;) {
+		struct listed_subvol subvol;
+
+		err = btrfs_util_subvolume_iterator_next_info(iter,
+							      &subvol.path,
+							      &subvol.info);
+		if (err == BTRFS_UTIL_ERROR_STOP_ITERATION) {
+			break;
+		} else if (err) {
+			error_btrfs_util(err);
+			goto out;
+		}
+
+		if (!filters_match(&subvol, filter_set)) {
+			free(subvol.path);
+			continue;
+		}
+
+		if (subvols->num >= capacity) {
+			struct subvol_list *new_subvols;
+			size_t new_capacity = max_t(size_t, 1, capacity * 2);
+
+			new_subvols = realloc(subvols,
+					      sizeof(*new_subvols) +
+					      new_capacity *
+					      sizeof(new_subvols->subvols[0]));
+			if (!new_subvols) {
+				error("out of memory");
+				goto out;
+			}
+
+			subvols = new_subvols;
+			capacity = new_capacity;
+		}
+
+		subvols->subvols[subvols->num] = subvol;
+		subvols->num++;
+	}
+
+	ret = 0;
+out:
+	if (iter)
+		btrfs_util_destroy_subvolume_iterator(iter);
+	if (ret) {
+		free_subvol_list(subvols);
+		subvols = NULL;
+	}
+	return subvols;
+}
+
+static int btrfs_list_subvols_print_v2(int fd,
+				    struct btrfs_list_filter_set_v2 *filter_set,
+				    struct btrfs_list_comparer_set_v2 *comp_set,
+				    enum btrfs_list_layout layout,
+				    int full_path, const char *raw_prefix)
+{
+	struct subvol_list *subvols;
+
+	/*
+	 * full_path hasn't done anything since 4f5ebb3ef553 ("Btrfs-progs: fix
+	 * to make list specified directory's subvolumes work"). See
+	 * https://www.spinics.net/lists/linux-btrfs/msg69820.html
+	 */
+
+	if (filter_set->only_deleted)
+		subvols = btrfs_list_deleted_subvols(fd, filter_set);
+	else
+		subvols = btrfs_list_subvols(fd, filter_set);
+	if (!subvols)
+		return -1;
+
+	sort_subvols(comp_set, subvols);
+
+	print_all_subvol_info(subvols, layout, raw_prefix);
+
+	free_subvol_list(subvols);
+
+	return 0;
+}
+
+static int btrfs_list_parse_sort_string_v2(char *opt_arg,
+					struct btrfs_list_comparer_set_v2 **comps)
+{
+	int order;
+	int flag;
+	char *p;
+	char **ptr_argv;
+	int what_to_sort;
+
+	while ((p = strtok(opt_arg, ",")) != NULL) {
+		flag = 0;
+		ptr_argv = all_sort_items;
+
+		while (*ptr_argv) {
+			if (strcmp(*ptr_argv, p) == 0) {
+				flag = 1;
+				break;
+			} else {
+				p++;
+				if (strcmp(*ptr_argv, p) == 0) {
+					flag = 1;
+					p--;
+					break;
+				}
+				p--;
+			}
+			ptr_argv++;
+		}
+
+		if (flag == 0)
+			return -1;
+
+		else {
+			if (*p == '+') {
+				order = 0;
+				p++;
+			} else if (*p == '-') {
+				order = 1;
+				p++;
+			} else
+				order = 0;
+
+			what_to_sort = btrfs_list_get_sort_item(p);
+			btrfs_list_setup_comparer_v2(comps,
+						what_to_sort, order);
+		}
+		opt_arg = NULL;
+	}
+
+	return 0;
+}
+
+static int btrfs_list_parse_filter_string_v2(char *opt_arg,
+					  struct btrfs_list_filter_set_v2 **filters,
+					  enum btrfs_list_filter_enum type)
+{
+
+	u64 arg;
+
+	switch (*(opt_arg++)) {
+	case '+':
+		arg = arg_strtou64(opt_arg);
+		type += 2;
+
+		btrfs_list_setup_filter_v2(filters, type, arg);
+		break;
+	case '-':
+		arg = arg_strtou64(opt_arg);
+		type += 1;
+
+		btrfs_list_setup_filter_v2(filters, type, arg);
+		break;
+	default:
+		opt_arg--;
+		arg = arg_strtou64(opt_arg);
+
+		btrfs_list_setup_filter_v2(filters, type, arg);
+		break;
+	}
+
+	return 0;
+}
+
 /*
  * Naming of options:
  * - uppercase for filters and sort options
@@ -451,8 +1360,8 @@ static const char * const cmd_subvol_list_usage[] = {
 
 static int cmd_subvol_list(int argc, char **argv)
 {
-	struct btrfs_list_filter_set *filter_set;
-	struct btrfs_list_comparer_set *comparer_set;
+	struct btrfs_list_filter_set_v2 *filter_set;
+	struct btrfs_list_comparer_set_v2 *comparer_set;
 	u64 flags = 0;
 	int fd = -1;
 	u64 top_id;
@@ -463,8 +1372,8 @@ static int cmd_subvol_list(int argc, char **argv)
 	DIR *dirstream = NULL;
 	enum btrfs_list_layout layout = BTRFS_LIST_LAYOUT_DEFAULT;
 
-	filter_set = btrfs_list_alloc_filter_set();
-	comparer_set = btrfs_list_alloc_comparer_set();
+	filter_set = btrfs_list_alloc_filter_set_v2();
+	comparer_set = btrfs_list_alloc_comparer_set_v2();
 
 	while(1) {
 		int c;
@@ -480,21 +1389,21 @@ static int cmd_subvol_list(int argc, char **argv)
 
 		switch(c) {
 		case 'p':
-			btrfs_list_setup_print_column(BTRFS_LIST_PARENT);
+			btrfs_list_setup_print_column_v2(BTRFS_LIST_PARENT);
 			break;
 		case 'a':
 			is_list_all = 1;
 			break;
 		case 'c':
-			btrfs_list_setup_print_column(BTRFS_LIST_OGENERATION);
+			btrfs_list_setup_print_column_v2(BTRFS_LIST_OGENERATION);
 			break;
 		case 'd':
-			btrfs_list_setup_filter(&filter_set,
+			btrfs_list_setup_filter_v2(&filter_set,
 						BTRFS_LIST_FILTER_DELETED,
 						0);
 			break;
 		case 'g':
-			btrfs_list_setup_print_column(BTRFS_LIST_GENERATION);
+			btrfs_list_setup_print_column_v2(BTRFS_LIST_GENERATION);
 			break;
 		case 'o':
 			is_only_in_path = 1;
@@ -503,27 +1412,27 @@ static int cmd_subvol_list(int argc, char **argv)
 			layout = BTRFS_LIST_LAYOUT_TABLE;
 			break;
 		case 's':
-			btrfs_list_setup_filter(&filter_set,
+			btrfs_list_setup_filter_v2(&filter_set,
 						BTRFS_LIST_FILTER_SNAPSHOT_ONLY,
 						0);
-			btrfs_list_setup_print_column(BTRFS_LIST_OGENERATION);
-			btrfs_list_setup_print_column(BTRFS_LIST_OTIME);
+			btrfs_list_setup_print_column_v2(BTRFS_LIST_OGENERATION);
+			btrfs_list_setup_print_column_v2(BTRFS_LIST_OTIME);
 			break;
 		case 'u':
-			btrfs_list_setup_print_column(BTRFS_LIST_UUID);
+			btrfs_list_setup_print_column_v2(BTRFS_LIST_UUID);
 			break;
 		case 'q':
-			btrfs_list_setup_print_column(BTRFS_LIST_PUUID);
+			btrfs_list_setup_print_column_v2(BTRFS_LIST_PUUID);
 			break;
 		case 'R':
-			btrfs_list_setup_print_column(BTRFS_LIST_RUUID);
+			btrfs_list_setup_print_column_v2(BTRFS_LIST_RUUID);
 			break;
 		case 'r':
 			flags |= BTRFS_ROOT_SUBVOL_RDONLY;
 			break;
 		case 'G':
-			btrfs_list_setup_print_column(BTRFS_LIST_GENERATION);
-			ret = btrfs_list_parse_filter_string(optarg,
+			btrfs_list_setup_print_column_v2(BTRFS_LIST_GENERATION);
+			ret = btrfs_list_parse_filter_string_v2(optarg,
 							&filter_set,
 							BTRFS_LIST_FILTER_GEN);
 			if (ret) {
@@ -533,8 +1442,8 @@ static int cmd_subvol_list(int argc, char **argv)
 			break;
 
 		case 'C':
-			btrfs_list_setup_print_column(BTRFS_LIST_OGENERATION);
-			ret = btrfs_list_parse_filter_string(optarg,
+			btrfs_list_setup_print_column_v2(BTRFS_LIST_OGENERATION);
+			ret = btrfs_list_parse_filter_string_v2(optarg,
 							&filter_set,
 							BTRFS_LIST_FILTER_CGEN);
 			if (ret) {
@@ -543,7 +1452,7 @@ static int cmd_subvol_list(int argc, char **argv)
 			}
 			break;
 		case 'S':
-			ret = btrfs_list_parse_sort_string(optarg,
+			ret = btrfs_list_parse_sort_string_v2(optarg,
 							   &comparer_set);
 			if (ret) {
 				uerr = 1;
@@ -571,7 +1480,7 @@ static int cmd_subvol_list(int argc, char **argv)
 	}
 
 	if (flags)
-		btrfs_list_setup_filter(&filter_set, BTRFS_LIST_FILTER_FLAGS,
+		btrfs_list_setup_filter_v2(&filter_set, BTRFS_LIST_FILTER_FLAGS,
 					flags);
 
 	ret = btrfs_list_get_path_rootid(fd, &top_id);
@@ -579,21 +1488,21 @@ static int cmd_subvol_list(int argc, char **argv)
 		goto out;
 
 	if (is_list_all)
-		btrfs_list_setup_filter(&filter_set,
+		btrfs_list_setup_filter_v2(&filter_set,
 					BTRFS_LIST_FILTER_FULL_PATH,
 					top_id);
 	else if (is_only_in_path)
-		btrfs_list_setup_filter(&filter_set,
+		btrfs_list_setup_filter_v2(&filter_set,
 					BTRFS_LIST_FILTER_TOPID_EQUAL,
 					top_id);
 
 	/* by default we shall print the following columns*/
-	btrfs_list_setup_print_column(BTRFS_LIST_OBJECTID);
-	btrfs_list_setup_print_column(BTRFS_LIST_GENERATION);
-	btrfs_list_setup_print_column(BTRFS_LIST_TOP_LEVEL);
-	btrfs_list_setup_print_column(BTRFS_LIST_PATH);
+	btrfs_list_setup_print_column_v2(BTRFS_LIST_OBJECTID);
+	btrfs_list_setup_print_column_v2(BTRFS_LIST_GENERATION);
+	btrfs_list_setup_print_column_v2(BTRFS_LIST_TOP_LEVEL);
+	btrfs_list_setup_print_column_v2(BTRFS_LIST_PATH);
 
-	ret = btrfs_list_subvols_print(fd, filter_set, comparer_set,
+	ret = btrfs_list_subvols_print_v2(fd, filter_set, comparer_set,
 			layout, !is_list_all && !is_only_in_path, NULL);
 
 out:
-- 
2.14.4

[PATCH v2 18/20] btrfs-progs: test: Add helper function to check if test user exists

From: Misono Tomohiro <hidden>
Date: 2018-06-18 08:38:19

Test user 'progs-test' will be used to test the behavior of normal user.

In order to pass this check, add the user by "useradd -M progs-test".
Note that progs-test should not have root privileges.

Signed-off-by: Misono Tomohiro <redacted>
---
 tests/common | 10 ++++++++++
 1 file changed, 10 insertions(+)
diff --git a/tests/common b/tests/common
index 7e4e09df..7a37a4cd 100644
--- a/tests/common
+++ b/tests/common
@@ -314,6 +314,16 @@ check_global_prereq()
 	fi
 }
 
+check_testuser()
+{
+	id -u progs-test > /dev/null 2>&1
+	if [ $? -ne 0 ]; then
+		_not_run "Need to add user \"progs-test\""
+	fi
+	# Note that progs-test should not have root privileges
+	# otherwise test may not run as expected
+}
+
 check_image()
 {
 	local image
-- 
2.14.4

Re: [PATCH v2 00/20] btrfs-progs: Rework of "subvolume list/show" and relax the root privileges of them

From: Misono Tomohiro <hidden>
Date: 2018-07-04 08:15:13

Gentle ping, as this is related to the new ioctls merged in 4.18-rc1.

On 2018/06/18 17:40, Misono Tomohiro wrote:
Changelog
 
 v1 -> v2: 
  generally update whole patch set, especially:
   - rebased to progs 4.17
   - Improve error handling
   - Update man/help/commit message
   - Add/Update several options of sub list:
      -f ... follow mounted subvolumes
      -a ... remove meaningless filter
      -A ... print path in absolute path
      --nosort ... output results incrementally
     Please see below examples
=====
github:  https://github.com/t-msn/btrfs-progs/tree/rework-sub-list

Hello,

This series requires some new ioctls which are now in kernel 4.18-rc1. 

The aim of this series is to relax the root privileges of "sub list/show"
while keeping as much output consistency between root and non-privileged
user. For "subvolume list", default output has been changed from current
btrfs-progs (in both old and new kernel) and some options are newly added.
For "subvolume show", root's output is the same as before but there are
some difference from non-privileged user's output. 

Please see below examples.


* Behavior summary of new "sub list"
  - default (no option)
    - lists subvolumes below the specified path (inc. path itself)
    - If new ioctls exists
      - the path can be non-subvolume directory
      - non-privileged user can call it
        (subvolumes to which the user cannot access will be skipped)

  - -f
    - follow mounted subvolume below the specified path and list them too 
      (only if it is the same filesystem)

  - -a
    - updated to remove filter. i.e. the output is the same as current progs
      without option (require root privileges)

  - -A
    - print path in absolute path

  -- nosort
    - output results incrementally without loading information to memory

 [Example]
  $ mkfs.btrfs -f $DEV
  $ mkfs.btrfs -f $DEV2
  $ mount $DEV $MNT

  $ btrfs subvolume create $MNT/AAA
  $ btrfs subvolume create $MNT/BBB
  $ btrfs subvolume create $MNT/CCC
  $ btrfs subvolume create $MNT/DDD
  $ mkdir $MNT/AAA/bbb
  $ mkdir $MNT/AAA/ccc
  $ mkdir $MNT/AAA/other

  $ umount $MNT
  $ mount -o subvol=AAA $DEV $MNT
  $ mount -o subvol=BBB $DEV $MNT/bbb
  $ mount -o subvol=CCC $DEV $MNT/ccc
  $ mount -o $DEV2 $MNT/other

  $ btrfs subvolume list $MNT # print subvolumes below the path
  ID 256 gen 10 top level 5 path .

  $ btrfs subvolume list -A $MNT # print path in absolute path
  ID 256 gen 10 top level 5 path /mnt

  $ btrfs subvolume list -f $MNT # follow mounted subvolumes too
  ID 256 gen 10 top level 5 path .
  ID 258 gen 7 top level 5 path bbb
  ID 259 gen 8 top level 5 path ccc

  $ btrfs subvolume list -a $MNT
  # print all subvolumes in the fs. same output as progs<=4.17 without option
  ID 256 gen 10 top level 5 path AAA
  ID 258 gen 7 top level 5 path BBB
  ID 259 gen 8 top level 5 path CCC
  ID 260 gen 9 top level 5 path DDD

 More details are in each commit log.


* Behavior summary of new "sub show"
  - No change for root's output
  - If new ioctls exists, non-privileged user can call it
    - In that case, path to be shown is absolute path
      (for root, it is relative to top-level subvolume)
      Also, snapshots to be shown are to which the user can
      access from current mount point.
      (for root, all snapshots in the fs)


* Patch structure
The first several patches update libbtrfsutil and the latter patches update
sub list/show command.

 1st patch is independent and updates man doc of btrfs-subvolume

 2nd-6th update the libbtrfsutil using new ioctls:
   - Relax the privileges of following functions if kernel supports new
     ioctls and @top/@id is zero (i.e. the given path/fd is used instead
     of arbitrary subvolume id).
     - util_subvolume_info()
     - subvolume iterator related ones (util_subvolume_iterator_next() etc.)

   - For subvolume iterator, if kernel supports new ioctls and @top is zero,
     non-subvolume directory can be specified as a start point. Also,
     subvolume which cannot be accessed (either because of permission
     error or not found (may happen if other volume is mounted in the
     path) will be skipped for non-privileged user.

   - Code path of root and non-privileged user is different. While root uses
     TREE_SEARCH ioctl as before, non-privileged user uses newly added
     ioctls. However, There is only one exception and when subvolume
     iterator is created from non-subvolume directory, code path of both is
     the same (and thus both use new ioctls).

 7th patch update the "sub list" to use libbtrfsutil (no behavior change yet)
   This is a copy of non-merged following patch originally written
   by Omar Sandoval:
     btrfs-progs: use libbtrfsutil for subvolume list [1]
   expect this commit keeps libbtrfs implementation which above commit
   tries to remove.

   (I suspect that the part of the reason that the original patch has not
   been merged is it removes libbtrfs and this commits modify this. but
   I'm completely fine with the original patch instead of this.)

 8th-15th patch update the behavior of "sub list"

 16th-17th patch update the behavior of "sub show"

 18th-20th patch are cli-test for "sub list" of new behavior.


* Future todo:
If this approach is ok, I'd like to update the output of "sub list" more like:
  - Remove obsolete field (i.e. top-level) from output

Any comments are welcome.
Thanks,
Tomohiro Misono

[1] https://www.spinics.net/lists/linux-btrfs/msg74917.html 

Misono Tomohiro (20):
  btrfs-progs: doc: Update man btrfs subvolume
  btrfs-progs: ioctl/libbtrfsutil: Add 3 definitions of new unprivileged
    ioctl
  btrfs-progs: libbtrfsutil: Factor out btrfs_util_subvolume_info_fd()
  btrfs-porgs: libbtrfsutil: Relax the privileges of
    util_subvolume_info()
  btrfs-progs: libbtrfsuitl: Factor out
    btrfs_util_subvolume_iterator_next()
  btrfs-progs: libbtrfsutil: Relax the privileges of subvolume iterator
  btrfs-progs: sub list: Use libbtrfsuitl for subvolume list
  btrfs-progs: sub list: factor out main part of btrfs_list_subvols
  btrfs-progs: sub list: Change the default behavior of "subvolume list"
    and allow non-privileged user to call it
  btrfs-progs: sub list: Add -A option to output path in absolute path
  btrfs-progs: sub list: Add -f option to follow mounted subvolumes
    below the path
  btrfs-progs: sub list: Add --nosort option to output incrementally
    without sort
  btrfs-progs: sub list: Update -a option and remove meaningless filter
  btrfs-progs: sub list: Update help message of -o option
  btrfs-progs: sub list: Update help message of -d option
  btrfs-progs: utils: Fallback to open without O_NOATIME flag in
    find_mount_root():
  btrfs-progs: sub show: Allow non-privileged user to call "subvolume
    show"
  btrfs-progs: test: Add helper function to check if test user exists
  btrfs-porgs: test: Add cli-test/009 to check subvolume list for both
    root and normal user
  btrfs-progs: test: Add cli-test/010 to check "subvolume list -f"
    option

 Documentation/btrfs-subvolume.asciidoc            |  108 +-
 cmds-subvolume.c                                  | 1372 ++++++++++++++++++++-
 ioctl.h                                           |   99 ++
 libbtrfsutil/btrfs.h                              |   97 ++
 libbtrfsutil/btrfsutil.h                          |   25 +-
 libbtrfsutil/errors.c                             |   10 +
 libbtrfsutil/subvolume.c                          |  494 +++++++-
 tests/cli-tests/009-subvolume-list/test.sh        |  134 ++
 tests/cli-tests/010-subvolume-list-follow/test.sh |   86 ++
 tests/common                                      |   10 +
 utils.c                                           |    3 +
 11 files changed, 2315 insertions(+), 123 deletions(-)
 create mode 100755 tests/cli-tests/009-subvolume-list/test.sh
 create mode 100755 tests/cli-tests/010-subvolume-list-follow/test.sh

Re: [PATCH v2 00/20] btrfs-progs: Rework of "subvolume list/show" and relax the root privileges of them

From: David Sterba <hidden>
Date: 2018-08-03 15:42:32

On Wed, Jul 04, 2018 at 05:14:59PM +0900, Misono Tomohiro wrote:
Gentle ping, as this is related to the new ioctls merged in 4.18-rc1.
Due to me spending more time than expected on kernel, this patchset will
be merged partially or the 4.18 will be deleayed.

Re: [PATCH v2 00/20] btrfs-progs: Rework of "subvolume list/show" and relax the root privileges of them

From: Misono Tomohiro <hidden>
Date: 2018-08-09 10:45:58

On 2018/08/03 22:46, David Sterba wrote:
On Wed, Jul 04, 2018 at 05:14:59PM +0900, Misono Tomohiro wrote:
quoted
Gentle ping, as this is related to the new ioctls merged in 4.18-rc1.
Due to me spending more time than expected on kernel, this patchset will
be merged partially or the 4.18 will be deleayed.
github version is rebased to 4.17.1:
  https://github.com/t-msn/btrfs-progs/tree/rework-sub-list

I will take a vacation from tomorrow and my response may be delayed.

Regards,
Misono

Re: [PATCH v2 00/20] btrfs-progs: Rework of "subvolume list/show" and relax the root privileges of them

From: David Sterba <hidden>
Date: 2018-08-16 00:50:10

Hi,

so I've merged the first bath to devel. There was quite some whitespace
damage and style issues, but I fixed that as I went through the pathes.

I've left out all patches that introduce new options, this is the hard
part and single letter options cannot be added just randomly. That's
what led to the current state.

The semantics of current options is preserved, with the slight exception
to the option -a.
* Future todo:
If this approach is ok, I'd like to update the output of "sub list" more like:
  - Remove obsolete field (i.e. top-level) from output
Yeah the top-level field can be removed from the user output. Current
idea of the fields printed by default is: id, generation, type, path.
The type is a representation of eg. read-write status,
subvolume/snapshot, and maybe received, or if it is default.

The remaining fields are on-demand by other options and the future proof
solution is to provide the single option to specify which columns to
print.

The inline format, ie. the current one, is not very comprehensive so the
table layout should be the default for user.

The path format needs to be selected by a more concise way than just -A
and possibly more added options. Eg. --path

* absolute (shorthand abs)
* realative to the current path (pwd)
* toplevel relative (toplevel)

In order to make it possible to distinguish in the output without prior
knowledge of which one was selected, I propose this formatting:

* absolute path always starts with / and is really absolute to the /
* relative path to the current directory starts with ./
* toplevel does not start with ./ nor /

There are patches touching the libbtrfsutil code, so I'd like to hear
from Omar what he thinks.

What's now in devel should not change and provide base for further
patches, so please use it if you're going to implement something from
the above.

Re: [PATCH v2 00/20] btrfs-progs: Rework of "subvolume list/show" and relax the root privileges of them

From: Misono Tomohiro <hidden>
Date: 2018-08-21 10:21:59

On 2018/08/16 3:12, David Sterba wrote:
Hi,

so I've merged the first bath to devel. There was quite some whitespace
damage and style issues, but I fixed that as I went through the pathes.
Thanks for taking this.
I've left out all patches that introduce new options, this is the hard
part and single letter options cannot be added just randomly. That's
what led to the current state.
Ok, but could you consider to add -f (11th path, or --follow instead of
short option) which is useful when btrfs is used for root fs
(or please remove cli-tests/010 for the moment from devel branch).
The semantics of current options is preserved, with the slight exception
to the option -a.
Devel branch also excludes 13th patch and behavior of -a is not changed,
is it intended? Without the patch, there is no way to get the same result
of current "btrfs subvol list" when mount point is not FS_TREE.
quoted
* Future todo:
If this approach is ok, I'd like to update the output of "sub list" more like:
  - Remove obsolete field (i.e. top-level) from output
Yeah the top-level field can be removed from the user output. Current
idea of the fields printed by default is: id, generation, type, path.
The type is a representation of eg. read-write status,
subvolume/snapshot, and maybe received, or if it is default.

The remaining fields are on-demand by other options and the future proof
solution is to provide the single option to specify which columns to
print.

The inline format, ie. the current one, is not very comprehensive so the
table layout should be the default for user.

The path format needs to be selected by a more concise way than just -A
and possibly more added options. Eg. --path

* absolute (shorthand abs)
* realative to the current path (pwd)
* toplevel relative (toplevel)

In order to make it possible to distinguish in the output without prior
knowledge of which one was selected, I propose this formatting:

* absolute path always starts with / and is really absolute to the /
* relative path to the current directory starts with ./
* toplevel does not start with ./ nor /

There are patches touching the libbtrfsutil code, so I'd like to hear
from Omar what he thinks.
Also, I want to get acknowledgement about 7th patch
(as I wrote in cover letter, I'm ok with original version too).
What's now in devel should not change and provide base for further
patches, so please use it if you're going to implement something from
the above
Understood. I will consider them if current patches in devel has been settled.

Regards,
Misono
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help