On Tue, Sep 11, 2018 at 12:40:36PM +0800, jianchao.wang wrote:

Hi Ming

On 09/11/2018 12:03 PM, Ming Lei wrote:

quoted

After the refcount is switched to atomic mode, the whole counting is
done on the atomic part. Then if the refcount need to switch to percpu mode
again, all percpu part of the counter is re-initialized as zero simply. This
is invariant with/without this patch.

Does the "whole counting" here means ?

(long)(sum (every cpu's refcounter)) + atomic refcounter

No.

We only check the 'whole counting' at atomic mode, so it is the atomic
part('ref->count'), please see percpu_ref_put_many(), in which ref->release()
is only called at atomic mode.

At percpu mode, the ref-count is only decreased/increased on the local CPU.

and when switch to atomic mode, there could be value left in atomic refcounter.
then the unsigned long percpu refcounter cound be decreased from 0.
	
From another angle, one request could be ended on a different cpu from the one where
it is issued.

Right.

But,

In the fast path(either completion or issue path), the .q_usage_counter is only
increased/decreased, which can be done in either atomic or percpu mode.

And when we want to check the queued/inflight requests, the .q_usage_counter has
to be killed first by switching to atomic mode, such as in all kinds of
uses of blk_freeze_queue_start().


Thanks,
Ming