On Thu, Oct 28, 2021 at 01:52:27PM +0800, JeffleXu wrote:

On 10/27/21 10:36 PM, Vivek Goyal wrote:

quoted

[snip]

quoted

Is the biggest issue the lack of visibility to see if the device supports DAX?

Not necessarily. I think for me two biggest issues are.

- Should dax be enabled by default in server as well. If we do that,
  server will have to make extra ioctl() call on every LOOKUP and GETATTR
  fuse request. Local filesystems probably can easily query FS_XFLAGS_DAX
  state but doing extra syscall all the time will probably be some cost
  (No idea how much).

I tested the time cost from virtiofsd's perspective (time cost of
passthrough_ll.c:lo_do_lookup()):
- before per inode DAX feature: 2~4 us
- after per inode DAX feature: 7~8 us

It is within expectation, as the introduction of per inode DAX feature,
one extra ioctl() system call is introduced.

Also the time cost from client's perspective (time cost of
fs/fuse/dir.c:fuse_lookup_name())
- before per inode DAX feature: 25~30 us
- after per inode DAX feature: 30~35 us

That is, ~15%~20% performance loss.

Currently we do ioctl() to query the persitent inode flags every time
FUSE_LOOKUP request is received, maybe we could cache the result of
ioctl() on virtiofsd side, but I have no idea how to intercept the
runtime modification to these persistent indoe flags from other
processes on host, e.g. sysadmin on host, to maintain the cache consistency.

Do you really expect the dax flag to change on individual files a lot?  This in
itself is an expensive operation as the FS has to flush the inode.

So if the default behavior of client side is 'dax=inode', and virtiofsd
disables per inode DAX by default (neither '-o dax=server|attr' is

I'm not following what dax=server or dax=attr is?

specified for virtiofsd) for the sake of performance, then guest won't
see DAX enabled and thus won't be surprised. This can reduce the
behavior change to the minimum.

What processes, other than virtiofsd have 'control' of these files?

I know that a sysadmin could come in and change the dax flag but I think that
is like saying a sys-admin can come in and change your .bashrc and your
environment is suddenly different.  We have to trust the admins not to do stuff
like that.  So I don't think admins are going to be changing the dax flag on
files out from under 'users'; in this case virtiofsd.  Right?

That means that virtiofsd could cache the status and avoid the performance
issues above correct?

Ira

quoted

- So far if virtiofs is mounted without any of the dax options, just
  by looking at mount option, I could tell, DAX is not enabled on any
  of the files. But that will not be true anymore. Because dax=inode
  be default, it is possible that server upgrade enabled dax on some
  or all the files.

  I guess I will have to stick to same reason given by ext4/xfs. That is
  to determine whether DAX is enabled on a file or not, you need to
  query STATX_ATTR_DAX flag. That's the only way to conclude if DAX is
  being used on a file or not. Don't look at filesystem mount options
  and reach a conclusion (except the case of dax=never).

-- 
Thanks,
Jeffle