* Peng Zhang [off-list ref] [230818 07:54]:

在 2023/8/17 02:30, Liam R. Howlett 写道:

quoted

* Peng Zhang [off-list ref] [230816 09:42]:

quoted

...

quoted

quoted

quoted

quoted

quoted

+/**
+ * __mt_dup(): Duplicate a maple tree
+ * @mt: The source maple tree
+ * @new: The new maple tree
+ * @gfp: The GFP_FLAGS to use for allocations
+ *
+ * This function duplicates a maple tree using a faster method than traversing
+ * the source tree and inserting entries into the new tree one by one. The user
+ * needs to lock the source tree manually. Before calling this function, @new
+ * must be an empty tree or an uninitialized tree. If @mt uses an external lock,
+ * we may also need to manually set @new's external lock using
+ * mt_set_external_lock().
+ *
+ * Return: 0 on success, -ENOMEM if memory could not be allocated.
+ */
+int __mt_dup(struct maple_tree *mt, struct maple_tree *new, gfp_t gfp)

We use mas_ for things that won't handle the locking and pass in a maple
state.  Considering the leaves need to be altered once this is returned,
I would expect passing in a maple state should be feasible?

But we don't really need mas here. What do you think the state of mas
should be when this function returns? Make it point to the first entry,
or the last entry?

I would write it to point to the first element so that the call to
replace the first element can just do that without an extra walk and
document the maple state end point.

Unfortunately, this does not seem to be convenient. Users usually use
mas_for_each() to replace elements. If we set mas to the first element,
the first call to mas_find() in mas_for_each() will get the next
element.

This sounds like the need for another iterator specifically for
duplicating.

quoted

There may also be other scenarios where the user does not necessarily
have to replace every element.

Do you mean a limit or elements that need to be skipped?  We could have
a limit on the iteration.

quoted

Finally, getting the first element in __mt_dup() requires an additional
check to check whether the first element has already been recorded. Such
a check will be performed at each leaf node, which is unnecessary
overhead.

Of course, the first reason is the main reason, which prevents us from
using mas_for_each(). So I don't want to record the first element.

I don't like the interface because it can easily be misunderstood and
used incorrectly.  I don't know how to make a cleaner interface, but
I've gone through a few thoughts:

The first was hide _all of it_ in a new iterator:
mas_dup_each(old, new, old_entry) {
	if (don't_dup(old_entry)) {
		mas_erase(new);
		continue;
	}

	mas_dup_insert(new, new_entry);
}

This iterator would check if mas_is_start(old) and dup the tree in that
event.  Leave the both new trees pointing to the first element and set
old_entry.  I don't know how to handle the failure in duplicating the
tree in this case - I guess we could return old_entry = NULL and check
if mas_is_err(old) after the loop.  Do you see a problem with this?

This interface looks OK. But handling the failure case is tricky.

I think it's awkward, but not tricky; possibly error prone to users.  I
don't like this solution because of the error handling.  I'm just
stating some options I ruled out in hopes you see some way of improving
them.

Maybe we name the check something like mas_dup_complete(old, new) and
all it does is checks for an error?  It makes it obvious that it's
necessary but doesn't avoid people leaving it out.

quoted

The second idea was an init of the old tree.  This is closest to what you
have:

if (mas_dup_init(old, new))
	goto -ENOMEM;

mas_dup_each(old, new) {
	if (don't_dup(old_entry)) {
		mas_erase(new);
		continue;
	}

	mas_dup_insert(new, new_entry);
}

I think this interface could be better.

quoted

This would duplicate the tree at the start and leave both pointing at
the first element so that mas_dup_each() could start on that element.
Each subsequent call would go to the next element in both maple states.

Every element of the new tree is the same as the old tree, and we don't
need to maintain the mas of the old tree. It is enough to maintain the
mas of the new tree when traversing.

Okay, and using DFS means we can't stop one level before the leaves
during duplication since deletes may not function.

quoted

It sounds like you don't want this for performance reasons?  Although

I mean I don't want to record the first element during duplicating. But
we can get the first element after the duplicate completes. This can
also still be within the implementation of the interface.

quoted

looking at mas_find() today, I think this could still work since we are
checking the maple state for a lot.

Yes, mas_find() does a whole bunch of checks.

quoted

Both ideas could be even faster than what you have if we handle the
special cases of mas_is_none()/mas_is_ptr() in a smarter way because we
don't need to be as worried about the entry point of the maple state as
much as we do with mas_find()/mas_for_each().  I mean, is it possible to
get to a mas_is_none() or mas_is_ptr() on duplicating a tree?  How do we
handle these users?

The check for mas_is_none() or mas_is_ptr() in mas_find() is really not
worth it if we hold the lock. There doesn't seem to be a good way around
mas_is_ptr() since it needs to enter the loop once. mas_is_none() can be
solved because it does not enter the loop, we can use it as a condition
to enter the loop.

So do you think it is worth making a new iterator then?  One that
requires the write lock so some work can be avoided?

Without using mas_find() to avoid the check inside, I have to figure out
how I can handle mas_is_ptr() properly.

quoted

Both ideas still suffer from someone saying "Gee, that {insert function
name here} is used in the forking code, so I can totally use that in my
code because that's how it work!"  and find out it works for the limited
testing they do.  Then it fails later and the emails start flying.


I almost think we should do something like this on insert:

void mas_dup_insert(old, new, new_entry) {
	WARN_ON_ONCE(old == new);
	WARN_ON_ONCE(old->index != new->index);
	WARN_ON_ONCE(old->last != new->last);
	...
}

Maintaining old mas doesn't feel worth it. If this we have to traverse
the old tree one more time.

Maybe only when debug is enabled?  We should at least keep the first
check.

I'm not committed to this interface either.  Do you have an idea that
works better?

quoted

This would at least _require_ someone to have two maple states and
hopefully think twice on using it where it should not be used.

The bottom line is that this code is close to what we need to make
forking better, but I fear the misuse of the interface.

Something else to think about:
In the work items for the Maple Tree, there is a plan to have an enum to
specify the type of write that is going to happen.  The idea was for
mas_preallocate() to set this type of write so we can just go right to
the correct function.  We could use that here and set the maple state
write type to a direct replacement.

This can be the next step. We can do without it for now.

Agreed, but your current replacement is very close to what we have for
direct replacement after all the checks in the current write path.

I'm wondering if we have an enum in the maple state to just go to that
store operation, then these two interfaces could share a lot of the
code.

But we should leave it for later, I was just pointing it out.

Thanks,
Liam