On Tue, Nov 2, 2021 at 1:09 PM Jan Kara [off-list ref] wrote:

On Wed 27-10-21 16:29:40, Vivek Goyal wrote:

quoted

On Wed, Oct 27, 2021 at 03:23:19PM +0200, Jan Kara wrote:

quoted

On Wed 27-10-21 08:59:15, Amir Goldstein wrote:

quoted

On Tue, Oct 26, 2021 at 10:14 PM Ioannis Angelakopoulos
[off-list ref] wrote:

quoted

On Tue, Oct 26, 2021 at 2:27 PM Vivek Goyal [off-list ref] wrote:
The problem here is that the OPEN event might still be travelling towards the guest in the
virtqueues and arrives after the guest has already deleted its local inode.
While the remote event (OPEN) received by the guest is valid, its fsnotify
subsystem will drop it since the local inode is not there.

I have a feeling that we are mixing issues related to shared server
and remote fsnotify.

I don't think Ioannis was speaking about shared server case here. I think
he says that in a simple FUSE remote notification setup we can loose OPEN
events (or basically any other) if the inode for which the event happens
gets deleted sufficiently early after the event being generated. That seems
indeed somewhat unexpected and could be confusing if it happens e.g. for
some directory operations.

Hi Jan,

Agreed. That's what Ioannis is trying to say. That some of the remote events
can be lost if fuse/guest local inode is unlinked. I think problem exists
both for shared and non-shared directory case.

With local filesystems we have a control that we can first queue up
the event in buffer before we remove local watches. With events travelling
from a remote server, there is no such control/synchronization. It can
very well happen that events got delayed in the communication path
somewhere and local watches went away and now there is no way to
deliver those events to the application.

So after thinking for some time about this I have the following question
about the architecture of this solution: Why do you actually have local
fsnotify watches at all? They seem to cause quite some trouble... I mean
cannot we have fsnotify marks only on FUSE server and generate all events
there? When e.g. file is created from the client, client tells the server
about creation, the server performs the creation which generates the
fsnotify event, that is received by the server and forwared back to the
client which just queues it into notification group's queue for userspace
to read it.

Now with this architecture there's no problem with duplicate events for
local & server notification marks, similarly there's no problem with lost
events after inode deletion because events received by the client are
directly queued into notification queue without any checking whether inode
is still alive etc. Would this work or am I missing something?

What about group #1 that wants mask A and group #2 that wants mask B
events?

Do you propose to maintain separate event queues over the protocol?
Attach a "recipient list" to each event?

I just don't see how this can scale other than:
- Local marks and connectors manage the subscriptions on local machine
- Protocol updates the server with the combined masks for watched objects

I think that the "post-mortem events" issue could be solved by keeping an
S_DEAD fuse inode object in limbo just for the mark.
When a remote server sends FS_IN_IGNORED or FS_DELETE_SELF for
an inode, the fuse client inode can be finally evicted.
I haven't tried to see how hard that would be to implement.

Thanks,
Amir.