On Tue, 16 May 2023 at 20:28, David Wysochanski [off-list ref] wrote:

On Tue, May 16, 2023 at 11:42 AM Chris Chilvers [off-list ref] wrote:

quoted

While testing the fscache performance fixes [1] that were merged into 6.4-rc1
it appears that the caching no longer works. The client will write to the cache
but never reads.

Thanks for the report.

If you reboot do you see reads from the cache?

On the first read after a reboot it uses the cache, but subsequent
reads do not use the cache.

You can check if the cache is being read from by looking in
/proc/fs/fscache/stats
at the "IO" line:
# grep IO /proc/fs/fscache/stats
IO     : rd=80030 wr=0

Running the tests 4 times (twice before reboot, and twice after) give
the following metrics:
FS-Cache I/O (delta)
Run       rd        wr
 1         0    39,250
 2       130    38,894
 3    39,000         0
 4        72    38,991

Can you share:
1. NFS server you're using (is it localhost or something else)
2. NFS version

The NFS server and client are separate VMs on the same network.
Server NFS version: Ubuntu 22.04 jammy, kernel 5.15.0-1021-gcp
Client NFS version: Ubuntu 22.04 jammy, kernel 6.4.0-060400rc1-generic
(https://kernel.ubuntu.com/~kernel-ppa/mainline/v6.4-rc1/)
Client nfs-utils: 2.6.3-rc6
Client cachefilesd: 0.10.10-0.2ubuntu1

In addition to checking the above for the reads from the cache, you can also
see whether NFS reads are going over the wire pretty easily with a similar
technique.

Copy /proc/self/mounstats to a file before your test, then make a second copy
after the test, then run mountstats as follows:
mountstats -S /tmp/mountstats.1 -f /tmp/mountstats.2

app read    = applications read bytes via read(2)
client read = client read bytes via NFS READ

Run           app read        client read
 1     322,122,547,200    322,122,547,200
 2     322,122,547,200    321,048,805,376
 3     322,122,547,200                  0
 4     322,122,547,200    321,593,053,184

I've put the full data in a GitHub gist, along with a graph collected
from a metrics agent:
https://gist.github.com/chilversc/54eb76155ad37b66cb85186e7449beaa
https://gist.githubusercontent.com/chilversc/54eb76155ad37b66cb85186e7449beaa/raw/09828c596d0cfc44bc0eb67f40e4033db202326e/metrics.png