On Jan 28, 2014, at 8:48 PM, Zheng Liu [off-list ref] wrote:

On Tue, Jan 28, 2014 at 03:35:14PM +0100, Jan Kara wrote:

quoted

On Tue 28-01-14 14:42:49, Zheng Liu wrote:

quoted

For project quota, the key issue is how to handle link(2)/rename(2).  We
summarize the behaviour in xfs as following.

*Note*
+ unaccounted dir
x accounted dir

link(2)
-------
		+		x
+		ok		error (EXDEV)
x		ok		error (EXDEV)

Presumably this accounted-to-accounted link() is only an error if
it is between directories of two different projects?

quoted

quoted

rename(2)
---------
		+		x
+		ok		ok
x		wrong		ok

 So moving unaccounted file/dir into an accounted dir would be OK? How is
that?

Actually xfs will return EXDEV error when we try to move unaccounted
file/dir into an accounted dir.  Then userspace tools (e.g. mv(1)) will
use create(2)/read(2)/write(2) syscalls to create these files/dirs from
scratch, and get the same id from their parent.

Why wouldn't renaming an unaccounted file into an accounted directory
just be implemented by doing the equivalent of chown() to change the
project ID and setting the quota?  That could avoid a HUGE amount of
data copying for large files.

So from the result we can see it is ok.  Quote from Dave Chinner's
comment: "that quota is accounted for when moving *into* an accounted
directory tree, not when moving out of a directory tree."

Sure, but IMHO returning -EXDEV in this case is a bit of a hack, and
increases the overhead of doing a rename within the filesystem a lot.

quoted

quoted

Further, project quota *cannot* be used with group quota at the same time.
On the other hand user quota and project quota can be used simultaneously.

 There's no fundamental reason for this and XFS folks actually recently
worked to remove this limitation. I don't think we should carry it over to
ext4.

Thanks for pointing it out.

quoted

quoted

2. http://xfs.org/index.php/XFS_FAQ#Q:_Quota:_What.27s_project_quota.3F

Design
======

Project id
----------
We have two options to store project id in inode.  a) define a new member
in ext4_inode structure; b) store project id in xattr.

Option a)
Pros:
 * Only need 4 bytes if we use a '__le32' type to store it

Cons:
 * Needs to change disk layout of ext4 inode

Option b)
Pros:
 * Don't need to change disk layout

Cons:
 * Take 24 bytes

 Cons of the b) is also that it's somewhat messier to get / set project id
from kernel. So I'm more in favor of a). I even think we could introduce
the additional id rather seamlessly using i_extra_i_size but I have to have
a look into details. Anyway I guess we can talk about the options at LSF.

I don't have a bias against both of two options.  It seems that we can
introduce a new id seamlessly using i_extra_isize.

1) old kernel + new disk layout
We can read/write new inode because new id doesn't be changed.

2) new kernel + old disk layout
We can use EXT4_FITS_IN_INODE to check whether new id can fit into an
inode or not.  We will check and report error when we try to enable
project quota on a file system with old disk layout in ext4_fill_super().

We also have a patch for e2fsck to increase i_extra_isize to ensure it
has enough space to hold a larger ext4_inode size, if this is required
for an existing filesystem that is upgraded to use this feature:

http://git.whamcloud.com/?p=tools/e2fsprogs.git;a=commit;h=e7653a1d3653d0bffc4617d8be8ce0a2c18b54c1

and tests for this feature:

http://git.whamcloud.com/?p=tools/e2fsprogs.git;a=commit;h=318a2688aa34e7dab383137fffaa413b882d13df

Cheers, Andreas

quoted

quoted

Here I propose to use option *b)* because it is easy for us to support
project id and we don't need to worry about changing disk layout.  But
I raise another issue here.  Now inline_data feature has been applied.
After waiting inline_data feature stable, we'd better enable inline_data
feature by default when we create a new ext4 file system.  Now the inode
size is 256 bytes by default, we have 72 bytes extra size to store
inline data:
 256 (default inode size) -
	156 (ext4_inode) + 4 (ext4_xattr_ibody_header) +
	20 (ext4_xattr_entry) + 4 (value) = 72

If we store project id in xattr, we just leave 48 bytes for inline data.
I am not sure whether or not it is too small for some users.

When we store project id in xattr, we will use {get,set}fattr to get/set
project id.  Thus we don't need to change userspace tool to manipulate
project id.  Meanwhile a _INHERENT flag for inode needs to be defined to
indicate that new directory creating in a directory with this flag will
get the same project id and get marked with this flag.  

Project quota API
-----------------
For keeping consistency with xfs, here I propose to use Q_X* flag to
communicate with kernel via quotactl(2) as we discussed.  Due to this we
need to define some callback functions to support Q_X* flag.  That means
that ext4 will support two quota flag sets for being compatible with
legacy userspace tools and use the same quotactl API to communicate with
kernel for project id like xfs.

 We can as well extend current VFS API to cover also project quotas. That
would make things somewhat more logical from userspace POV. 

Your meaning is that we support Q_* flag and Q_X* flag simultaneously?

Thanks,
                                               - Zheng

quoted

quoted

Currently quota subsystem in vfs doesn't handle project quota.  Thus we
need to make quota subsystem handle project id properly (e.g.
dquot_transfer, dquot_initialize).  We need to define a new callback
function in order to get project id.  Now in vfs we can access uid/gid
directly from inode, but we have no way to get project id.  A generic
callback function is defined to handle uid/gid.  The file system itself
can handle project id.  Until now only ext4 needs to implement this
callback function by itself because xfs doesn't use vfs quota subsystem.

 So we need to get ids from external structures only in two places. One is
dquot_initialize() and the other is dquot_transfer(). Instead of providing
callback to get project id, we could just create a variant of these functions
which will get required ids from a passed array instead of directly from
the inode.

quoted

For handling link(2)/rename(2) like xfs, we only allow hard link or
rename operation when the project ids are the same.  Otherwise we will
return EXDEV error to notify the user.

Quota-tools
-----------
Now quota-tools (e.g. quotaon, edquota, etc...) don't support project
quota.  Thus we need to make it support project id.  I believe that Li
Xi did some works on quota-tools.

E2fsprogs
---------
After supporting project quota, we need to change e2fsck(1) to make sure
that all sub-directories with _INHERENT flag have the same project id.
Meanwhile we need to make chattr(1) set/clear _INHERENT flag.

								Honza
-- 
Jan Kara [off-list ref]
SUSE Labs, CR

Cheers, Andreas