From: Jiri Kosina <redacted>
There are possibilities [1] how mincore() could be used as a converyor of
a sidechannel information about pagecache metadata.
Provide vm.mincore_privileged sysctl, which makes it possible to mincore()
start returning -EPERM in case it's invoked by a process lacking
CAP_SYS_ADMIN.
The default behavior stays "mincore() can be used by anybody" in order to
be conservative with respect to userspace behavior.
[1] https://www.theregister.co.uk/2019/01/05/boffins_beat_page_cache/
Signed-off-by: Jiri Kosina <redacted>
---
Documentation/sysctl/vm.txt | 9 +++++++++
kernel/sysctl.c | 8 ++++++++
mm/mincore.c | 5 +++++
3 files changed, 22 insertions(+)
@@ -41,6 +41,7 @@ Currently, these files are in /proc/sys/vm: - min_free_kbytes - min_slab_ratio - min_unmapped_ratio+- mincore_privileged - mmap_min_addr - mmap_rnd_bits - mmap_rnd_compat_bits
@@ -485,6 +486,14 @@ files and similar are considered. The default is 1 percent. ==============================================================+mincore_privileged:++mincore() could be potentially used to mount a side-channel attack against+pagecache metadata. This sysctl provides system administrators means to+make it available only to processess that own CAP_SYS_ADMIN capability.++The default is 0, which means mincore() can be used without restrictions.+============================================================== mmap_min_addr
@@ -114,6 +114,7 @@ extern unsigned int sysctl_nr_open_min, sysctl_nr_open_max;#ifndef CONFIG_MMUexternintsysctl_nr_trim_pages;#endif+externintsysctl_mincore_privileged;/* Constants used for minimum and maximum */#ifdef CONFIG_LOCKUP_DETECTOR
From: Jiri Kosina <redacted>
There are possibilities [1] how mincore() could be used as a converyor of
a sidechannel information about pagecache metadata.
Provide vm.mincore_privileged sysctl, which makes it possible to mincore()
start returning -EPERM in case it's invoked by a process lacking
CAP_SYS_ADMIN.
Haven't checked the details yet, but wouldn't it be safe if anonymous private
mincore() kept working, and restrictions were applied only to page cache?
The default behavior stays "mincore() can be used by anybody" in order to
be conservative with respect to userspace behavior.
What if we lied instead of returned -EPERM, to not break userspace so obviously?
I guess false positive would be the safer lie?
@@ -41,6 +41,7 @@ Currently, these files are in /proc/sys/vm: - min_free_kbytes - min_slab_ratio - min_unmapped_ratio+- mincore_privileged - mmap_min_addr - mmap_rnd_bits - mmap_rnd_compat_bits
@@ -485,6 +486,14 @@ files and similar are considered. The default is 1 percent. ==============================================================+mincore_privileged:++mincore() could be potentially used to mount a side-channel attack against+pagecache metadata. This sysctl provides system administrators means to+make it available only to processess that own CAP_SYS_ADMIN capability.++The default is 0, which means mincore() can be used without restrictions.+============================================================== mmap_min_addr
@@ -114,6 +114,7 @@ extern unsigned int sysctl_nr_open_min, sysctl_nr_open_max;#ifndef CONFIG_MMUexternintsysctl_nr_trim_pages;#endif+externintsysctl_mincore_privileged;/* Constants used for minimum and maximum */#ifdef CONFIG_LOCKUP_DETECTOR
There are possibilities [1] how mincore() could be used as a converyor of
a sidechannel information about pagecache metadata.
Provide vm.mincore_privileged sysctl, which makes it possible to mincore()
start returning -EPERM in case it's invoked by a process lacking
CAP_SYS_ADMIN.
Haven't checked the details yet, but wouldn't it be safe if anonymous private
mincore() kept working, and restrictions were applied only to page cache?
I was considering that, but then I decided not to do so, as that'd make
the interface even more confusing and semantics non-obvious in the
'privileged' case.
quoted
The default behavior stays "mincore() can be used by anybody" in order to
be conservative with respect to userspace behavior.
What if we lied instead of returned -EPERM, to not break userspace so
obviously? I guess false positive would be the safer lie?
So your proposal basically would be
if (privileged && !CAP_SYS_ADMIN)
if (pagecache)
return false;
else
return do_mincore()
right ?
I think userspace would hate us for that semantics, but on the other hand
I can sort of understand the 'mincore() is racy anyway, so what' argument,
if that's what you are suggesting.
But then, I have no idea what userspace is using mincore() for.
https://codesearch.debian.net/search?q=mincore might provide some insight
I guess (thanks Matthew).
--
Jiri Kosina
SUSE Labs
There are possibilities [1] how mincore() could be used as a converyor of
a sidechannel information about pagecache metadata.
Provide vm.mincore_privileged sysctl, which makes it possible to mincore()
start returning -EPERM in case it's invoked by a process lacking
CAP_SYS_ADMIN.
Haven't checked the details yet, but wouldn't it be safe if anonymous private
mincore() kept working, and restrictions were applied only to page cache?
I was considering that, but then I decided not to do so, as that'd make
the interface even more confusing and semantics non-obvious in the
'privileged' case.
quoted
quoted
The default behavior stays "mincore() can be used by anybody" in order to
be conservative with respect to userspace behavior.
What if we lied instead of returned -EPERM, to not break userspace so
obviously? I guess false positive would be the safer lie?
So your proposal basically would be
if (privileged && !CAP_SYS_ADMIN)
if (pagecache)
return false;
I was thinking about "return true" here, assuming that userspace generally wants
to ensure itself there won't be page faults when it starts doing something
critical, and if it sees a "false" it will try to do some kind of prefaulting,
possibly in a loop. There might be somebody trying to make sure something is out
of pagecache (it wants to see "false"), but can't think of anything except
benchmarks?
else
return do_mincore()
right ?
I think userspace would hate us for that semantics, but on the other hand
I can sort of understand the 'mincore() is racy anyway, so what' argument,
if that's what you are suggesting.
But then, I have no idea what userspace is using mincore() for.
https://codesearch.debian.net/search?q=mincore might provide some insight
I guess (thanks Matthew).
I was thinking about "return true" here, assuming that userspace generally wants
to ensure itself there won't be page faults when it starts doing something
critical, and if it sees a "false" it will try to do some kind of prefaulting,
possibly in a loop. There might be somebody trying to make sure something is out
Isn't that racy by design as the pages may get flushed out after the check?
Shouldn't the application use e.g. mlock()/.... to guarantee no page
faults in the first place?
MfG,
Bernd
--
Bernd Petrovitsch Email : bernd@petrovitsch.priv.at
LUGA : http://www.luga.at
Shouldn't the application use e.g. mlock()/.... to guarantee no page
faults in the first place?
Calling mincore() on pages you've just mlock()ed is sort of pointless
though.
Obviously;-)
Sorry for being unclear above: If I want my application to
avoid suffering from page faults, I use simply mlock()
(and/or friends) to nail the relevant pages into physical
RAM and not "look if they are out, if yes, get them in" which
has also the risk that these important pages are too soon
evicted again.
But perhaps I'm missing some very special use cases ....
MfG,
Brend
--
Bernd Petrovitsch Email : bernd@petrovitsch.priv.at
LUGA : http://www.luga.at
From: Kirill A. Shutemov <hidden> Date: 2019-01-08 14:09:26
On Tue, Jan 08, 2019 at 02:53:00PM +0100, Bernd Petrovitsch wrote:
On 08/01/2019 12:37, Jiri Kosina wrote:
quoted
On Tue, 8 Jan 2019, Bernd Petrovitsch wrote:
quoted
Shouldn't the application use e.g. mlock()/.... to guarantee no page
faults in the first place?
Calling mincore() on pages you've just mlock()ed is sort of pointless
though.
Obviously;-)
Sorry for being unclear above: If I want my application to
avoid suffering from page faults, I use simply mlock()
(and/or friends) to nail the relevant pages into physical
RAM and not "look if they are out, if yes, get them in" which
has also the risk that these important pages are too soon
evicted again.
Note, that mlock() doesn't prevent minor page faults. Mlocked memory is
still subject to mechanisms that makes the page temporary unmapped. For
instance migration (including NUMA balancing), compaction, khugepaged...
--
Kirill A. Shutemov
On Sat, Jan 5, 2019 at 9:27 AM Jiri Kosina [off-list ref] wrote:
From: Jiri Kosina <redacted>
There are possibilities [1] how mincore() could be used as a converyor of
a sidechannel information about pagecache metadata.
Can we please just limit it to vma's that are either anonymous, or map
a file that the user actually owns?
Then the capability check could be for "override the file owner check"
instead, which makes tons of sense.
No new sysctl's for something like this, please.
Linus
There are possibilities [1] how mincore() could be used as a converyor of
a sidechannel information about pagecache metadata.
Can we please just limit it to vma's that are either anonymous, or map
a file that the user actually owns?
Then the capability check could be for "override the file owner check"
instead, which makes tons of sense.
Makes sense.
I am still not completely sure what to return in such cases though; we can
either blatantly lie and always pretend that the pages are resident (to
avoid calling process entering some prefaulting mode), or return -ENOMEM
for mappings of files that don't belong to the user (in case it's not
CAP_SYS_ADMIN one).
--
Jiri Kosina
SUSE Labs
[ Crossed emails ]
On Sat, Jan 5, 2019 at 12:12 PM Jiri Kosina [off-list ref] wrote:
I am still not completely sure what to return in such cases though; we can
either blatantly lie and always pretend that the pages are resident
That's what my untested patch did. Or maybe just claim they are all
not present?
And again, that patch was entirely untested, so it may be garbage and
have some fundamental problem. I also don't know exactly what rule
might make most sense, but "you can write to the file" certainly to me
implies that you also could know what parts of it are in-core.
Who actually _uses_ mincore()? That's probably the best guide to what
we should do. Maybe they open the file read-only even if they are the
owner, and we really should look at file ownership instead.
I tried to make that "can_do_mincore()" function easy to understand
and easy to just modify to some sane state.
Again, my patch is meant as a "perhaps something like this?" rather
than some "this is _exactly_ how it must be done". Take the patch as a
quick suggestion, not some final answer.
Linus
I am still not completely sure what to return in such cases though; we can
either blatantly lie and always pretend that the pages are resident
That's what my untested patch did. Or maybe just claim they are all
not present?
Thinking about it a little bit more, I believe Vlastimil has a good point
with 'non present' potentially causing more bogus activity in userspace in
response (in an effort to actually make them present, and failing
indefinitely).
IOW, I think it's a reasonable expectation that the common scenario is
"check if it's present, and if not, try to fault it in" instead of "check
if it's present, and if it is, try to evict it".
And again, that patch was entirely untested, so it may be garbage and
have some fundamental problem.
I will be travelling for next ~24 hours, but I have just asked our QA guys
to run it through some basic battery of testing (which will probably
happen on monday anyway).
I also don't know exactly what rule might make most sense, but "you can
write to the file" certainly to me implies that you also could know what
parts of it are in-core.
I think it's reasonable; I can't really imagine any sidechannel to a
global state be possibly mounted on valid R/W mappings. I'd guess that
probably the most interesting here are the code segments of shared
libraries, allowing to trace victim's execution.
Who actually _uses_ mincore()? That's probably the best guide to what
we should do. Maybe they open the file read-only even if they are the
owner, and we really should look at file ownership instead.
On Sat, Jan 5, 2019 at 12:43 PM Jiri Kosina [off-list ref] wrote:
quoted
Who actually _uses_ mincore()? That's probably the best guide to what
we should do. Maybe they open the file read-only even if they are the
owner, and we really should look at file ownership instead.
Yeah, heh.
And the first hit is 'fincore', which probably nobody cares about
anyway, but it does
fd = open (name, O_RDONLY)
..
mmap(window, len, PROT_NONE, MAP_PRIVATE, ..
so if we want to keep that working, we'd really need to actually check
file ownership rather than just looking at f_mode.
But I don't know if anybody *uses* and cares about fincore, and it's
particularly questionable for non-root users.
And the Android go runtime code seems to oddly use mincore to figure
out page size:
// try using mincore to detect the physical page size.
// mincore should return EINVAL when address is not a multiple of
system page size.
which is all kinds of odd, but whatever.. Why mincore, rather than
something sane and obvious like mmap? Don't ask me...
Anyway, the Debian code search just results in mostly non-present
stuff. It's sad that google code search is no more. It was great for
exactly these kinds of questions.
The mono runtime seems to have some mono_pages_not_faulted() function,
but I don't know if people use it for file mappings, and I couldn't
find any interesting users of it.
I didn't find anything that seems to really care, but I gave up after
a few pages of really boring stuff.
Linus
From: Kevin Easton <hidden> Date: 2019-01-06 11:33:45
On Sat, Jan 05, 2019 at 01:54:03PM -0800, Linus Torvalds wrote:
On Sat, Jan 5, 2019 at 12:43 PM Jiri Kosina [off-list ref] wrote:
quoted
quoted
Who actually _uses_ mincore()? That's probably the best guide to what
we should do. Maybe they open the file read-only even if they are the
owner, and we really should look at file ownership instead.
Anyway, the Debian code search just results in mostly non-present
stuff. It's sad that google code search is no more. It was great for
exactly these kinds of questions.
The mono runtime seems to have some mono_pages_not_faulted() function,
but I don't know if people use it for file mappings, and I couldn't
find any interesting users of it.
I didn't find anything that seems to really care, but I gave up after
a few pages of really boring stuff.
Linus
From: Kevin Easton <hidden> Date: 2019-01-08 08:51:05
On Sat, Jan 05, 2019 at 01:54:03PM -0800, Linus Torvalds wrote:
On Sat, Jan 5, 2019 at 12:43 PM Jiri Kosina [off-list ref] wrote:
quoted
quoted
Who actually _uses_ mincore()? That's probably the best guide to what
we should do. Maybe they open the file read-only even if they are the
owner, and we really should look at file ownership instead.
Yeah, heh.
And the first hit is 'fincore', which probably nobody cares about
anyway, but it does
fd = open (name, O_RDONLY)
..
mmap(window, len, PROT_NONE, MAP_PRIVATE, ..
so if we want to keep that working, we'd really need to actually check
file ownership rather than just looking at f_mode.
But I don't know if anybody *uses* and cares about fincore, and it's
particularly questionable for non-root users.
...
I didn't find anything that seems to really care, but I gave up after
a few pages of really boring stuff.
I've gone through everything in the Debian code search, and this is the
stuff that seems like it would be affected at all by the current patch:
util-linux
Contains 'fincore' as already noted above.
e2fsprogs
e4defrag tries to drop pages that it caused to be loaded into the
page cache, but it's not clear that this ever worked as designed
anyway (it calls mincore() before ioctl(fd, EXT4_IOC_MOVE_EXT ..)
but then after the sync_file_range it drops the pages that *were*
in the page cache at the time of mincore()).
pgfincore
postgresql extension used to try to dump/restore page cache status
of database backing files across reboots. It uses a fresh mapping
with mincore() to try to determine the current page cache status of
a file.
nocache
LD_PRELOAD library that tries to drop any pages that the victim
program has caused to be loaded into the page cache, uses mincore
on a fresh mapping to see what was resident beforehand. Also
includes 'cachestats' command that's basically another 'fincore'.
xfsprogs
xfs_io has a 'mincore' sub-command that is roughly equivalent to
'fincore'.
vmtouch
vmtouch is "Portable file system cache diagnostics and control",
among other things it implements 'fincore' type functionality, and
one of its touted use-cases is "Preserving virtual memory profile
when failing over servers".
qemu
qemu uses mincore() with a fresh PROT_NONE, MAP_PRIVATE mapping to
implement the "x-check-cache-dropped" option.
( https://patchwork.kernel.org/patch/10395865/ )
(Everything else I could see was either looking at anonymous VMAs, its
own existing mapping that it's been using for actual IO, or was just
using mincore() to see if an address was part of any mapping at all).
- Kevin
Hello, Linus.
On Sat, Jan 05, 2019 at 01:54:03PM -0800, Linus Torvalds wrote:
And the first hit is 'fincore', which probably nobody cares about
anyway, but it does
fd = open (name, O_RDONLY)
..
mmap(window, len, PROT_NONE, MAP_PRIVATE, ..
So, folks here have been using fincore(1) for diagnostic purposes and
are also looking to expand on it to investigate per-cgroup cache
usages (mmap -> mincore -> /proc/self/pagemap -> /proc/kpagecgroup ->
cgroup path).
These are all root-only usages to find out what's going on with the
whole page cache. We aren't attached to doing things this particular
way but it'd suck if there's no way.
Thanks.
--
tejun
On Sat, Jan 5, 2019 at 6:27 PM Jiri Kosina [off-list ref] wrote:
There are possibilities [1] how mincore() could be used as a converyor of
a sidechannel information about pagecache metadata.
Provide vm.mincore_privileged sysctl, which makes it possible to mincore()
start returning -EPERM in case it's invoked by a process lacking
CAP_SYS_ADMIN.
The default behavior stays "mincore() can be used by anybody" in order to
be conservative with respect to userspace behavior.
[1] https://www.theregister.co.uk/2019/01/05/boffins_beat_page_cache/
Just checking: I guess /proc/$pid/pagemap (iow, the pagemap_read()
handler) is less problematic because it only returns data about the
state of page tables, and doesn't query the address_space? In other
words, it permits monitoring evictions, but non-intrusively detecting
that something has been loaded into memory by another process is
harder?
On Sat, Jan 5, 2019 at 2:55 PM Jann Horn [off-list ref] wrote:
Just checking: I guess /proc/$pid/pagemap (iow, the pagemap_read()
handler) is less problematic because it only returns data about the
state of page tables, and doesn't query the address_space? In other
words, it permits monitoring evictions, but non-intrusively detecting
that something has been loaded into memory by another process is
harder?
Yes. I think it was brought up during the original report, but to use
the pagemap for this, you'd basically need to first populate all the
pages, and then wait for pageout.
So pagemap *does* leak very similar data, but it's much harder to use
as an attack vector.
That said, I think "mincore()" is just the simple one. You *can* (and
this was also discussed on the security list) do things like
- fault in a single page
- the kernel will opportunistically fault in pages that it already
has available _around_ that page.
- then use pagemap (or just _timing_ - no real kernel support needed)
to see if those pages are now mapped in your address space
so honestly, mincore is just the "big hammer" and easy way to get at
some of this data. But it's probably worth closing exactly because
it's easy. There are other ways to get at the "are these pages mapped"
information, but they are a lot more combersome to use.
Side note: maybe we could just remove the "__mincore_unmapped_range()"
thing entirely, and basically make mincore() do what pagemap does,
which is to say "are the pages mapped in this VM".
That would be nicer than my patch, simply because removing code is
always nice. And arguably it's a better semantic anyway.
Linus
On Sat, Jan 5, 2019 at 3:05 PM Linus Torvalds
[off-list ref] wrote:
That would be nicer than my patch, simply because removing code is
always nice. And arguably it's a better semantic anyway.
Yeah, I wonder why we did that thing where mincore() walks the page
tables, but if they are empty it looks in the page cache.
[... goes and looks in history ..]
It goes back to forever, it looks like. I can't find a reason.
Anyway, a removal patch would look something like the attached, I
think. That makes mincore() actually say how many pages are in _this_
mapping, not how many pages could be paged in without doing IO.
Hmm. Maybe we should try this first. Simplicity is always good.
Again, obviously untested.
Linus
On Sat, Jan 5, 2019 at 3:16 PM Linus Torvalds
[off-list ref] wrote:
It goes back to forever, it looks like. I can't find a reason.
Our man-pages talk abouit the "without doing IO" part. That may be the
result of our code, though, not the reason for it.
The BSD man-page has other flags, but doesn't describe what "in core"
really means:
MINCORE_INCORE Page is in core (resident).
MINCORE_REFERENCED Page has been referenced by us.
MINCORE_MODIFIED Page has been modified by us.
MINCORE_REFERENCED_OTHER Page has been referenced.
MINCORE_MODIFIED_OTHER Page has been modified.
MINCORE_SUPER Page is part of a large (``super'') page.
but the fact that it has MINCORE_MODIFIED_OTHER does obviously imply
that yes, historically it really did look up the pages elsewhere, not
just in the page tables.
Still, maybe we can get away with just making it be about our own page
tables. That would be lovely.
Linus
On Sat, Jan 5, 2019 at 3:16 PM Linus Torvalds
[off-list ref] wrote:
It goes back to forever, it looks like. I can't find a reason.
mincore() was originally added in 2.3.52pre3, it looks like. Around
2000 or so. But sadly before the BK history.
And that comment about
"Later we can get more picky about what "in core" means precisely."
that still exists above mincore_page() goes back to the original patch.
Linus
From: Matthew Wilcox <willy@infradead.org> Date: 2019-01-06 00:11:46
On Sat, Jan 05, 2019 at 03:39:10PM -0800, Linus Torvalds wrote:
On Sat, Jan 5, 2019 at 3:16 PM Linus Torvalds
[off-list ref] wrote:
quoted
It goes back to forever, it looks like. I can't find a reason.
mincore() was originally added in 2.3.52pre3, it looks like. Around
2000 or so. But sadly before the BK history.
And that comment about
"Later we can get more picky about what "in core" means precisely."
that still exists above mincore_page() goes back to the original patch.
FreeBSD claims to have a manpage from SunOS 4.1.3 with mincore (!)
https://www.freebsd.org/cgi/man.cgi?query=mincore&apropos=0&sektion=0&manpath=SunOS+4.1.3&arch=default&format=html
DESCRIPTION
mincore() returns the primary memory residency status of pages in the
address space covered by mappings in the range [addr, addr + len). The
status is returned as a char-per-page in the character array referenced
by *vec (which the system assumes to be large enough to encompass all
the pages in the address range). The least significant bit of each
character is set to 1 to indicate that the referenced page is in pri-
mary memory, 0 if it is not. The settings of other bits in each char-
acter is undefined and may contain other information in the future.
It's still not clear that "primary memory residency status" actually means.
Does it mean "mapped", or does it mean "exists in caches and doesn't need IO".
I don't even know what kind of caches SunOS 4.1.3 had. The Linux
implementation depends on the page cache, and wouldn't work (at least
not very well) in a system that has a traditional disk buffer cache.
Anyway, I guess it's mostly moot. From a "does this cause regressions"
standpoint, the only thing that matters is really whatever Linux
programs that have used this since it was introduced 18+ years ago.
But I think my patch to just rip out all that page lookup, and just
base it on the page table state has the fundamental advantage that it
gets rid of code. Maybe I should jst commit it, and see if anything
breaks? We do have options in case things break, and then we'd at
least know who cares (and perhaps a lot more information of _why_ they
care).
Linus
On Sat, Jan 5, 2019 at 4:22 PM Linus Torvalds
[off-list ref] wrote:
But I think my patch to just rip out all that page lookup, and just
base it on the page table state has the fundamental advantage that it
gets rid of code. Maybe I should jst commit it, and see if anything
breaks? We do have options in case things break, and then we'd at
least know who cares (and perhaps a lot more information of _why_ they
care).
Slightly updated patch in case somebody wants to try things out.
Also, any comments about the pmd_trans_unstable() case?
Linus
On Sat, Jan 5, 2019 at 5:50 PM Linus Torvalds
[off-list ref] wrote:
Slightly updated patch in case somebody wants to try things out.
I decided to just apply that patch. It is *not* marked for stable,
very intentionally, because I expect that we will need to wait and see
if there are issues with it, and whether we might have to do something
entirely different (more like the traditional behavior with some extra
"only for owner" logic).
But doing a test patch during the merge window (which is about to
close) sounds like the right thing to do.
Linus
From: Dave Chinner <david@fromorbit.com> Date: 2019-01-08 04:43:46
On Sun, Jan 06, 2019 at 01:46:37PM -0800, Linus Torvalds wrote:
On Sat, Jan 5, 2019 at 5:50 PM Linus Torvalds
[off-list ref] wrote:
quoted
Slightly updated patch in case somebody wants to try things out.
I decided to just apply that patch. It is *not* marked for stable,
very intentionally, because I expect that we will need to wait and see
if there are issues with it, and whether we might have to do something
entirely different (more like the traditional behavior with some extra
"only for owner" logic).
So, I read the paper and before I was half way through it I figured
there are a bunch of other similar page cache invalidation attacks
we can perform without needing mincore. i.e. Focussing on mmap() and
mincore() misses the wider issues we have with global shared caches.
My first thought:
fd = open(some_file, O_RDONLY);
iov[0].iov_base = buf;
iov[0].iov_len = 1;
ret = preadv2(fd, iov, 1, off, RWF_NOWAIT);
switch (ret) {
case -EAGAIN:
/* page not resident in page cache */
break;
case 1:
/* page resident in page cache */
break;
default:
/* beyond EOF or some other error */
break;
}
This is "as documented" in the man page for preadv2:
RWF_NOWAIT (since Linux 4.14)
Do not wait for data which is not immediately available.
If this flag is specified, the preadv2() system call will
return instantly if it would have to read data from the
backing storage or wait for a lock. If some data was
successfully read, it will return the number of bytes read.
If no bytes were read, it will return -1 and set errno to
EAGAIN. Currently, this flag is meaningful only for
preadv2().
IOWs, we've explicitly designed interfaces to communicate whether
data is "immediately accessible" or not to the application so they
can make sensible decisions about IO scheduling. i.e. IO won't
block the main application processing loop and so can be scheduled in
the background by the app and the data processed when that IO
returns. That just so happens to be exactly the same information
about the page cache that mincore is making available to userspace.
If we "remove" this information from the interfaces like it has been
done for mincore(), it doesn't mean userspace can't get it in other
ways. e.g. it now just has to time the read(2) syscall duration and
infer whether the data came from the page cache or disk from the
timing information.
IMO, there's nothing new or novel about this page cache information
leak - it was just a matter of time before some researcher put 2 and
2 together and realised that sharing the page cache across a
security boundary is little different from sharing deduplicated
pages across those same security boundaries. i.e. As long as we
shared caches across security boundaries and userspace can control
both cache invalidation and instantiation, we cannot prevent
userspace from constructing these invalidate+read information
exfiltration scenarios.
And then there is overlayfs. Overlay is really just a way to
efficiently share the page cache of the same underlying read-only
directory tree across all containers on a host. i.e. we have been
specifically designing our container hosting systems to share the
underlying read-only page cache across all security boundaries on
the host. If overlay is vulnerable to these shared page cache
attacks (seems extremely likely) then we've got much bigger problems
than mincore to think about....
But doing a test patch during the merge window (which is about to
close) sounds like the right thing to do.
IMO it seems like the wrong thing to do. It's just a hacky band-aid
over a specific extraction method and does nothing to reduce the
actual scope of the information leak. Focussing on the band-aid
means you've missed all the other avenues that the same information
is exposed and all the infrastructure we've build on the core
concept of sharing kernel side pages across security boundaries.
And that's even without considering whether the change breaks
userspace. Which it does. e.g. vmtouch is fairly widely used to
manage page cache instantiation for rapid bring-up and migration of
guest VMs and containers. They save the hot page cache information
from a running container and then using that to instantiate the page
cache in new instances running the same workload so they run at full
speed right from the start. This use case calls mincore() to pull
the page cache information from the running container.
If anyone else proposed merging a syscall implementation change that
was extremely likely to break userspace you'd be shouting at them
that "we don't break userspace"....
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
On Mon, Jan 7, 2019 at 8:43 PM Dave Chinner [off-list ref] wrote:
So, I read the paper and before I was half way through it I figured
there are a bunch of other similar page cache invalidation attacks
we can perform without needing mincore. i.e. Focussing on mmap() and
mincore() misses the wider issues we have with global shared caches.
Oh, agreed, and that was discussed in the original report too.
The thing is, you can also depend on our pre-faulting of pages in the
page fault handler, and use that to get the cached status of nearby
pages. So do something like "fault one page, then do mincore() to see
how many pages near it were mapped". See our "do_fault_around()"
logic.
But mincore is certainly the easiest interface, and the one that
doesn't require much effort or setup. It's also the one where our old
behavior was actually arguably simply stupid and actively wrong (ie
"in caches" isn't even strictly speaking a valid question, since the
caches in question may be invalid). So let's try to see if giving
mincore() slightly more well-defined semantics actually causes any
pain.
I do think that the RWF_NOWAIT case might also be interesting to look at.
Linus
From: Dave Chinner <david@fromorbit.com> Date: 2019-01-09 02:24:37
On Tue, Jan 08, 2019 at 09:57:49AM -0800, Linus Torvalds wrote:
On Mon, Jan 7, 2019 at 8:43 PM Dave Chinner [off-list ref] wrote:
quoted
So, I read the paper and before I was half way through it I figured
there are a bunch of other similar page cache invalidation attacks
we can perform without needing mincore. i.e. Focussing on mmap() and
mincore() misses the wider issues we have with global shared caches.
Oh, agreed, and that was discussed in the original report too.
The thing is, you can also depend on our pre-faulting of pages in the
page fault handler, and use that to get the cached status of nearby
pages. So do something like "fault one page, then do mincore() to see
how many pages near it were mapped". See our "do_fault_around()"
logic.
Observing fault-around could help you detect what code an application is
running, but it's not necessary (and can be turned off). Also, such
an it observation is not dependent on using mincore. neither
fault-around nor mincore are required functionality to exploit the
information leaks.
And, FWIW, fault-around actually destroys the information in the
exfiltration channel described in the paper because it perturbs the
carefully constructed page cache residency pattern that encodes the
message.
But mincore is certainly the easiest interface, and the one that
doesn't require much effort or setup.
Off the top of my head, here's a few vectors for reading the page
cache residency state without perturbing the page cache residency
pattern:
- mincore
- preadv2(RWF_NOWAIT)
- fadvise(POSIX_FADV_RANDOM); timed read(2) syscalls
- madvise(MADV_RANDOM); timed read of first byte in each page
i.e. mincore is a messenger, but it's not the only trivial
observation technique available. The only difference between mincore
and the others will be the observation latency and hence channel
bandwidth.
IOWs, the question we need to focus on now is not "does breaking
mincore affect anyone", it is "how the hell do we mitigate and
isolate an information leak exposed by fundamental OS functionality
that *everything* depends on for performance"?
It's also the one where our old
behavior was actually arguably simply stupid and actively wrong (ie
"in caches" isn't even strictly speaking a valid question, since the
caches in question may be invalid).
This is irrelevant to the problem reported. Sure, mincore may be
an awful interface, but it's semantics are not the cause of the
information leak. You're just shooting the messenger...
I do think that the RWF_NOWAIT case might also be interesting to look at.
As are all the other mechanisms you can use to observer page cache
residency without perturbing it's state.
Keep in mind that the researchers documented a remote observation
technique that leaked the information across the network to a remote
host, so this leak has much, much wider scope than changing mincore
can address...
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
But mincore is certainly the easiest interface, and the one that
doesn't require much effort or setup.
Off the top of my head, here's a few vectors for reading the page
cache residency state without perturbing the page cache residency
pattern:
- mincore
- preadv2(RWF_NOWAIT)
- fadvise(POSIX_FADV_RANDOM); timed read(2) syscalls
- madvise(MADV_RANDOM); timed read of first byte in each page
While I obviously agree that all those are creating pagecache sidechannel
in principle, I think we really should mostly focus on the first two (with
mincore() already having been covered).
Rationale has been provided by Daniel Gruss in this thread -- if the
attacker is left with cache timing as the only available vector, he's
going to be much more successful with mounting hardware cache timing
attack anyway.
Thanks,
--
Jiri Kosina
SUSE Labs
From: Dave Chinner <david@fromorbit.com> Date: 2019-01-09 04:39:14
On Wed, Jan 09, 2019 at 03:31:35AM +0100, Jiri Kosina wrote:
On Wed, 9 Jan 2019, Dave Chinner wrote:
quoted
quoted
But mincore is certainly the easiest interface, and the one that
doesn't require much effort or setup.
Off the top of my head, here's a few vectors for reading the page
cache residency state without perturbing the page cache residency
pattern:
- mincore
- preadv2(RWF_NOWAIT)
- fadvise(POSIX_FADV_RANDOM); timed read(2) syscalls
- madvise(MADV_RANDOM); timed read of first byte in each page
While I obviously agree that all those are creating pagecache sidechannel
in principle, I think we really should mostly focus on the first two (with
mincore() already having been covered).
FWIW, I just realised that the easiest, most reliable way to
invalidate the page cache over a file range is simply to do a
O_DIRECT read on it. IOWs, all three requirements of this
information leak - highly specific, reliable cache invalidation
control, controlled cache instantiation and 3rd-party detection of
cache residency can all be performed with just the read(2)
syscall...
Rationale has been provided by Daniel Gruss in this thread -- if the
attacker is left with cache timing as the only available vector, he's
going to be much more successful with mounting hardware cache timing
attack anyway.
No, he said:
"Restricting mincore() is sufficient to fix the hardware-agnostic
part."
That's not correct - preadv2(RWF_NOWAIT) is also hardware agnostic
and provides exactly the same information about the page cache as
mincore. Timed read/mmap access loops for cache observation are
also hardware agnostic, and on fast SSD based storage will only be
marginally slower bandwidth than preadv2(RWF_NOWAIT).
Attackers will pick whatever leak vector we don't fix, so we either
fix them all (which I think is probably impossible without removing
caching altogether) or we start thinking about how we need to
isolate the page cache so that information isn't shared across
important security boundaries (e.g. page cache contents are
per-mount namespace).
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
FWIW, I just realised that the easiest, most reliable way to invalidate
the page cache over a file range is simply to do a O_DIRECT read on it.
Neat, good catch indeed. Still, it's only the invalidation part, but the
residency check is the crucial one.
quoted
Rationale has been provided by Daniel Gruss in this thread -- if the
attacker is left with cache timing as the only available vector, he's
going to be much more successful with mounting hardware cache timing
attack anyway.
No, he said:
"Restricting mincore() is sufficient to fix the hardware-agnostic
part."
That's not correct - preadv2(RWF_NOWAIT) is also hardware agnostic and
provides exactly the same information about the page cache as mincore.
Yeah, preadv2(RWF_NOWAIT) is in the same teritory as mincore(), it has
"just" been overlooked. I can't speak for Daniel, but I believe he might
be ok with rephrasing the above as "Restricting mincore() and RWF_NOWAIT
is sufficient ...".
Timed read/mmap access loops for cache observation are also hardware
agnostic, and on fast SSD based storage will only be marginally slower
bandwidth than preadv2(RWF_NOWAIT).
Attackers will pick whatever leak vector we don't fix, so we either fix
them all (which I think is probably impossible without removing caching
altogether)
We can't really fix the fact that it's possible to do the timing on the HW
caches though.
or we start thinking about how we need to isolate the page cache so that
information isn't shared across important security boundaries (e.g. page
cache contents are per-mount namespace).
Umm, sorry for being dense, but how would that help that particular attack
scenario on a system that doesn't really employ any namespacing? (which I
still believe is a majority of the systems out there, but I might have
just missed the containers train long time ago :) ).
--
Jiri Kosina
SUSE Labs
From: Dave Chinner <david@fromorbit.com> Date: 2019-01-10 01:15:41
On Wed, Jan 09, 2019 at 11:08:57AM +0100, Jiri Kosina wrote:
On Wed, 9 Jan 2019, Dave Chinner wrote:
quoted
FWIW, I just realised that the easiest, most reliable way to invalidate
the page cache over a file range is simply to do a O_DIRECT read on it.
Neat, good catch indeed. Still, it's only the invalidation part, but the
residency check is the crucial one.
quoted
quoted
Rationale has been provided by Daniel Gruss in this thread -- if the
attacker is left with cache timing as the only available vector, he's
going to be much more successful with mounting hardware cache timing
attack anyway.
No, he said:
"Restricting mincore() is sufficient to fix the hardware-agnostic
part."
That's not correct - preadv2(RWF_NOWAIT) is also hardware agnostic and
provides exactly the same information about the page cache as mincore.
Yeah, preadv2(RWF_NOWAIT) is in the same teritory as mincore(), it has
"just" been overlooked. I can't speak for Daniel, but I believe he might
be ok with rephrasing the above as "Restricting mincore() and RWF_NOWAIT
is sufficient ...".
Good luck with restricting RWF_NOWAIT. I eagerly await all the
fstests that exercise both the existing and new behaviours to
demonstrate they work correctly.
quoted
Timed read/mmap access loops for cache observation are also hardware
agnostic, and on fast SSD based storage will only be marginally slower
bandwidth than preadv2(RWF_NOWAIT).
Attackers will pick whatever leak vector we don't fix, so we either fix
them all (which I think is probably impossible without removing caching
altogether)
We can't really fix the fact that it's possible to do the timing on the HW
caches though.
We can't really fix the fact that it's possible to do the timing on
the page cache, either.
quoted
or we start thinking about how we need to isolate the page cache so that
information isn't shared across important security boundaries (e.g. page
cache contents are per-mount namespace).
Umm, sorry for being dense, but how would that help that particular attack
scenario on a system that doesn't really employ any namespacing?
What's your security boundary?
The "detect what code an app is running" exploit is based on
invalidating and then observing how shared, non-user-owned files
mapped with execute privileges change cache residency.
If the security boundary is within the local container, should users
inside that container be allowed to invalidate the cache of
executable files and libraries they don't own? In this case, we
can't stop observation, because that only require read permissions
and high precision timing, hence the only thing that can be done
here is prevent non-owners from invalidating the page cache.
If the security boundary is a namespace or guest VM, then permission
checks don't work - the user may own the file within that container.
This problem now is that the page cache is observable and
controllable from both sides of the fence. Hence the only way to
prevent observation of the code being run in a different namespace
is to prevent the page being shared across both containers.
The exfiltration exploit requires the page cache to be observable
and controllable on both sides of the security boundary. Should
users be able to observe and control the cached pages accessed by a
different container? KSM page deduplication lessons say no. This is
an even harder problem, because page cache residency can be observed
from remote machines....
What scares me is that new features being proposed could make our
exposure a whole lot worse. e.g. the recent virtio-pmem ("fake-dax")
proposal will directly share host page cache pages into guest VMs w/
DAX capability. i.e. the guest directly accesses the host page
cache. This opens up the potential for host page cache timing
attacks from the guest VMs, and potential guest to guest
observation/exploitation is possible if the same files are mapped
into multiple guests....
IOws the two questions here are simply: "What's your security
boundary?" and "Is the page cache visible and controllable on both
sides?".
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
Yeah, preadv2(RWF_NOWAIT) is in the same teritory as mincore(), it has
"just" been overlooked. I can't speak for Daniel, but I believe he might
be ok with rephrasing the above as "Restricting mincore() and RWF_NOWAIT
is sufficient ...".
Good luck with restricting RWF_NOWAIT. I eagerly await all the
fstests that exercise both the existing and new behaviours to
demonstrate they work correctly.
Well, we can still resurrect my original aproach of doing this opt-in
based on a sysctl setting, and letting the admin choose his poison.
If 'secure' mode is selected, RWF_NOWAIT will then probably just always
fail wit EAGAIN.
--
Jiri Kosina
SUSE Labs
On Tue, Jan 8, 2019 at 8:39 PM Dave Chinner [off-list ref] wrote:
FWIW, I just realised that the easiest, most reliable way to
invalidate the page cache over a file range is simply to do a
O_DIRECT read on it.
If that's the case, that's actually an O_DIRECT bug.
It should only invalidate the caches on write.
On reads, it wants to either _flush_ any direct caches before the
read, or just take the data from the caches. At no point is
"invalidate" a valid model.
Of course, I'm not in the least bit shocked if O_DIRECT is buggy like
this. But looking at least at the ext4 routine, the read just does
ret = filemap_write_and_wait_range(mapping, iocb->ki_pos,
and I don't see any invalidation.
Having read access to a file absolutely should *not* mean that you can
flush caches on it. That's a write op.
Any filesystem that invalidates the caches on read is utterly buggy.
Can you actually point to such a thing? Let's get that fixed, because
it's completely wrong regardless of this whole mincore issue.
Linus
From: Dave Chinner <david@fromorbit.com> Date: 2019-01-10 00:44:31
On Wed, Jan 09, 2019 at 10:25:43AM -0800, Linus Torvalds wrote:
On Tue, Jan 8, 2019 at 8:39 PM Dave Chinner [off-list ref] wrote:
quoted
FWIW, I just realised that the easiest, most reliable way to
invalidate the page cache over a file range is simply to do a
O_DIRECT read on it.
If that's the case, that's actually an O_DIRECT bug.
It should only invalidate the caches on write.
Sounds nice from a theoretical POV, but reality has taught us
very different lessons.
FWIW, a quick check of XFS's history so you understand how long this
behaviour has been around. It was introduced in the linux port in
2001 as direct IO support was being added:
commit e837eac23662afae603aaaef7c94bc839c1b8f67
Author: Steve Lord [off-list ref]
Date: Mon Mar 5 16:47:52 2001 +0000
Add bounds checking for direct I/O, do the cache invalidation for
data coherency on direct I/O.
This was basically a direct port of the flush+invalidation code in
the Irix direct IO path, which was introduced in 1995:
> revision 1.149
> date: 1995/08/11 20:09:44; author: ajs; state: Exp; lines: +70 -2
> 280514 Adding page cache flusing calls to make direct
> I/O coherent with buffered I/O.
IOWs, history tells us that invalidation for direct IO reads has
been done on XFS for almost 25 years. I know for certain that there
have been applications out there that depend on this
invalidation-on-read behaviour (another of those "reality bites"
lessons) so we can't just remove it because you *think* it is a bug.
i.e. we *could* remove the invalidation on read, but this we have a
major behavioural change to the XFS direct IO path. This means we
need to determine if we've just awoken sleeping data corruption
krakens as well as determine if there are any performance
regressions that result from the behavioural change.
Which brings me to validation. If the recent
clone/dedupe/copy_file_range() debacle has taught me anything, it's
that validating a "simple" IO path mechanism is going to take months
worth of machine time before we have any confidence that the change
is not going to expose users to new data corruption problems.
That's the difficulty here - it only takes 5 minutes to change
the code, but there's months of machine time needed to determine if
it's really safe to make that code change. Testing has a nasty habit
of finding invalid assumptions; when those are assumptions about
data coherency and integrity we can't test them on our users.
And, really, this would be just another band-aid over a symptom of
the information leak - it doesn't prevent users from being able to
control page cache invalidation. It just removes one method, just
like hacking mincore only removes one method of observing the page
cache. And, like mincore(), there's every chance it impacts on
userspace in a negative manner and so we need to be very careful
here.
On reads, it wants to either _flush_ any direct caches before the
read, or just take the data from the caches. At no point is
"invalidate" a valid model.
Of course, I'm not in the least bit shocked if O_DIRECT is buggy like
this. But looking at least at the ext4 routine, the read just does
ret = filemap_write_and_wait_range(mapping, iocb->ki_pos,
and I don't see any invalidation.
I wouldn't look at ext4 as an example of a reliable, problem free
direct IO implementation because, historically speaking, it's been a
series of nasty hacks (*cough* mount -o dioread_nolock *cough*) and
been far worse than XFS from data integrity, performance and
reliability perspectives.
IMO, "because ext4" has been a poor reason for justifying anything
for a long time, not the least when talking about features that
didn't even originate in extN....
Can you actually point to such a thing? Let's get that fixed, because
it's completely wrong regardless of this whole mincore issue.
The biggest problem that remains today is that we have no mechanism
for serialising page faults against DIO. If we leave pages cached in
memory while we have a AIO+DIO read (or write!) in progress, we can
dirty the page and run a buffered read before the AIO+DIO read
returns. This now leaves us in the state where where the AIO+DIO
read returns different (stale) data to a buffered read that has
already completed because it hit the dirty page cache. i.e. we
still have nasty page cache vs direct IO coherency problems, and
they are largely unsolvable because of the limitations of the core
kernel infrastructure architecture.
Yes, you can argue that userspace is doing an insane thing, but
every so often we come across coherency issues like this that are
out of a user's control (e.g. backup scan vs app accesses) and we do
our best to ensure that they don't cause problems given the
constraints we have. Invalidating the page cache on dio reads
mostly mitigates these coherency race conditions and that's why it's
still there in the XFS code paths...
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
On Wed, Jan 9, 2019 at 4:44 PM Dave Chinner [off-list ref] wrote:
I wouldn't look at ext4 as an example of a reliable, problem free
direct IO implementation because, historically speaking, it's been a
series of nasty hacks (*cough* mount -o dioread_nolock *cough*) and
been far worse than XFS from data integrity, performance and
reliability perspectives.
That's some big words from somebody who just admitted to much worse hacks.
Seriously. XFS is buggy in this regard, ext4 apparently isn't.
Thinking that it's better to just invalidate the cache for direct IO
reads is all kinds of odd.
Linus
From: Andy Lutomirski <luto@kernel.org> Date: 2019-01-10 05:26:57
On Wed, Jan 9, 2019 at 5:18 PM Linus Torvalds
[off-list ref] wrote:
On Wed, Jan 9, 2019 at 4:44 PM Dave Chinner [off-list ref] wrote:
quoted
I wouldn't look at ext4 as an example of a reliable, problem free
direct IO implementation because, historically speaking, it's been a
series of nasty hacks (*cough* mount -o dioread_nolock *cough*) and
been far worse than XFS from data integrity, performance and
reliability perspectives.
That's some big words from somebody who just admitted to much worse hacks.
Seriously. XFS is buggy in this regard, ext4 apparently isn't.
Thinking that it's better to just invalidate the cache for direct IO
reads is all kinds of odd.
This whole discussion seems to have gone a little bit off the rails...
Linus, I think I agree with Dave's overall sentiment, though, and I
think you should consider reverting your patch. Here's why. The
basic idea behind the attack is that the authors found efficient ways
to do two things: evict a page from page cache and detect, *without
filling the cache*, whether a page is cached. The combination lets an
attacker efficiently tell when another process reads a page. We need
to keep in mind that this attack is a sophisticated attack, and anyone
using it won't have any problem using a nontrivial way to detect
whether a page is in page cache.
So, unless we're going to try for real to make it hard to tell whether
a page is cached without causing that page to become cached, it's not
worth playing whack-a-mole. And, right now, mincore is whacking a
mole. RWF_NOWAIT appears to do essentially the same thing at very
little cost. I haven't really dug in, but I assume that various
prefaulting tricks combined with various pagetable probing tricks can
do similar things, but that's at least a *lot* more complicated.
So unless we're going to lock down RWF_NOWAIT as well, I see no reason
to lock down mincore(). Direct IO is a red herring -- O_DIRECT is
destructive enough that it seems likely to make the attacks a lot less
efficient.
--- begin digression ---
Since direct IO has been brought up, I have a question. I've wondered
for years why direct IO works the way it does. If I were implementing
it from scratch, my first inclination would be to use the page cache
instead of fighting it. To do a single-page direct read, I would look
that page up in the page cache (i.e. i_pages these days). If the page
is there, I would do a normal buffered read. If the page is not
there, I would insert a record into i_pages indicating that direct IO
is in progress and then I would do the IO into the destination page.
If any other read, direct or otherwise, sees a record saying "under
direct IO", it would wait.
To do a single-page direct write, I would look it up in i_pages. If
it's there, I would do a buffered write followed by a sync (because
applications expect a sync). If it's not there, I would again add a
record saying "under direct IO" and do the IO.
The idea is that this works as much like buffered IO as possible,
except that the pages backing the IO aren't normal sharable page cache
pages.
The multi-page case would be just an optimization on top of the
single-page case. The idea would be to somehow mark i_pages with
entire extents under direct IO. It's a radix tree -- this can, at
least in theory, be done efficiently. As long as all direct IO
operations run in increasing order of offset, there shouldn't be lock
ordering problems.
Other than history and possibly performance, is there any reason that
direct IO doesn't work this way?
P.S. What, if anything, prevents direct writes from causing trouble
when the underlying FS or backing store needs stable pages?
Similarly, what, if anything, prevents direct reads from temporarily
exposing unintended data to user code if the fs or underlying device
transforms the data during the read process (e.g. by decrypting
something)?
From: Matthew Wilcox <willy@infradead.org> Date: 2019-01-10 14:47:20
On Wed, Jan 09, 2019 at 09:26:41PM -0800, Andy Lutomirski wrote:
Since direct IO has been brought up, I have a question. I've wondered
for years why direct IO works the way it does. If I were implementing
it from scratch, my first inclination would be to use the page cache
instead of fighting it. To do a single-page direct read, I would look
that page up in the page cache (i.e. i_pages these days). If the page
is there, I would do a normal buffered read. If the page is not
there, I would insert a record into i_pages indicating that direct IO
is in progress and then I would do the IO into the destination page.
If any other read, direct or otherwise, sees a record saying "under
direct IO", it would wait.
OK, you're in the same ballpark I am ;-) Kent Overstreet pointed out
that what you want to do here is great for the mixed case, but it's
pretty inefficient for IOs to files which are wholly uncached.
So what I'm currently thinking about is an rwsem which works like this:
O_DIRECT task:
if i_pages is empty, take rwsem for read, recheck i_pages is empty, do IO,
drop rwsem.
if i_pages is not empty, insert XA_LOCK_ENTRY, when IO complete, wake waitqueue for that (mapping, index).
buffered IO:
if i_pages is empty, take rwsem for write, allocate page, insert page, drop rwsem.
if i_pages is not empty, look up index, if entry is XA_LOCK_ENTRY sleep on
waitqueue. otherwise proceed as now.
From: Dave Chinner <david@fromorbit.com> Date: 2019-01-10 21:44:40
On Thu, Jan 10, 2019 at 06:47:11AM -0800, Matthew Wilcox wrote:
On Wed, Jan 09, 2019 at 09:26:41PM -0800, Andy Lutomirski wrote:
quoted
Since direct IO has been brought up, I have a question. I've wondered
for years why direct IO works the way it does. If I were implementing
it from scratch, my first inclination would be to use the page cache
instead of fighting it. To do a single-page direct read, I would look
that page up in the page cache (i.e. i_pages these days). If the page
is there, I would do a normal buffered read. If the page is not
there, I would insert a record into i_pages indicating that direct IO
is in progress and then I would do the IO into the destination page.
If any other read, direct or otherwise, sees a record saying "under
direct IO", it would wait.
OK, you're in the same ballpark I am ;-) Kent Overstreet pointed out
that what you want to do here is great for the mixed case, but it's
pretty inefficient for IOs to files which are wholly uncached.
So what I'm currently thinking about is an rwsem which works like this:
O_DIRECT task:
if i_pages is empty, take rwsem for read, recheck i_pages is empty, do IO,
drop rwsem.
GUP does page fault on user buffer which is a mmapped region of same
file. page fault sets up for buffered IO, tries to take rwsem for
write, deadlocks.
Most of the schemes we come up with fall down at this point - you
can't hold a lock over gup that is also used in the buffered IO
path. That's why XFS (and now ext4) have the IOLOCK and MMAPLOCK
for truncation serialisation - we can't lock out both read()/write()
and mmap IO paths with the same lock...
if i_pages is not empty, insert XA_LOCK_ENTRY, when IO complete, wake waitqueue for that (mapping, index).
I assume you really mean add a tag to the entry?
But this means there is no record ofthe direct IO being in flight
except for the rwsem being held across the IO. Even if we did insert
a flag to say "DIO in progress" and not rely on the lock....
buffered IO:
if i_pages is empty, take rwsem for write, allocate page, insert page, drop rwsem.
if i_pages is not empty, look up index, if entry is XA_LOCK_ENTRY sleep on
waitqueue. otherwise proceed as now.
... we'll sleep on that flags in the page fault and deadlock anyway.
I'm pretty sure we explored this "record DIO state in the radix
tree" 2 or 3 years ago and came to the conclusion that it didn't
work for reasons like the above. i.e. it doesn't solve the problems
we currently have with locking and serialisation between DIO and
mmap...
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
On Thu, Jan 10, 2019 at 1:44 PM Dave Chinner [off-list ref] wrote:
GUP does page fault on user buffer which is a mmapped region of same
file. page fault sets up for buffered IO, tries to take rwsem for
write, deadlocks.
Most of the schemes we come up with fall down at this point - you
can't hold a lock over gup that is also used in the buffered IO
path. That's why XFS (and now ext4) have the IOLOCK and MMAPLOCK
for truncation serialisation - we can't lock out both read()/write()
and mmap IO paths with the same lock...
Side note: a somewhat similar version of is true even in the absence
of GUP and dio, for the case of doing a mmap of a file, and then
reading or writing from the mapped region into the file itself.
There are "interesting" locking scenarios wrt just holding the page
locked, and trying to then fill that page with information with just a
regular "copy_from_user()".
Page fault -> try to read the file -> oops, the page we're trying to
read from is locked because we're trying to write to it.
So we have that odd dance in generic_perform_write() which does
- touch the first user byte without holding any lock
- do write_begin() (which gets the page lock)
- copy from user space using the "atomic" copy (which just gives up on fault)
- if nothing got copied, go back and try again with a smaller copy
that can't cross a page. We might have raced with pageout.
It might be possible to do something similar for direct IO, although
simpler: just do the GUP entirely atomically (and in the fault case
just fall back to non-direct IO).
Linus
From: Dave Chinner <david@fromorbit.com> Date: 2019-01-11 01:47:34
On Wed, Jan 09, 2019 at 09:26:41PM -0800, Andy Lutomirski wrote:
Since direct IO has been brought up, I have a question. I've wondered
for years why direct IO works the way it does. If I were implementing
it from scratch, my first inclination would be to use the page cache
instead of fighting it. To do a single-page direct read, I would look
that page up in the page cache (i.e. i_pages these days). If the page
is there, I would do a normal buffered read. If the page is not
Therein lies the problem. Copying data is prohibitively expensive,
and that's the primary reason for O_DIRECT existing. i.e. O_DIRECT
is a low-overhead, zero-copy data movement interface.
The moment we switch from using CPU to dispatch IO to copying data,
performance goes down because we will be unable to keep storage
pipelines full. IOWs, any rework of O_DIRECT that involves copying
data is a non-starter.
But let's bring this back to the issue at hand - observability of
page cache residency of file pages. If th epage is caceh resident,
then it will have a latency of copying that data out of the page
(i.e. very low latency). If the page is not resident, then it will
do IO and take much, much longer to complete. i.e. we have clear
timing differences between cachce hit and cache miss IO. This is
exactly the timing information needed for observing page cache
residency.
We need to work out how to make page cache residency less
observable, not add new, near perfect observation mechanisms that
third parties can easily exploit...
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
From: Dave Chinner <david@fromorbit.com> Date: 2019-01-10 07:04:08
On Wed, Jan 09, 2019 at 05:18:21PM -0800, Linus Torvalds wrote:
On Wed, Jan 9, 2019 at 4:44 PM Dave Chinner [off-list ref] wrote:
quoted
I wouldn't look at ext4 as an example of a reliable, problem free
direct IO implementation because, historically speaking, it's been a
series of nasty hacks (*cough* mount -o dioread_nolock *cough*) and
been far worse than XFS from data integrity, performance and
reliability perspectives.
That's some big words from somebody who just admitted to much worse hacks.
Sorry, what hacks did I just admit to making? This O_DIRECT
behaviour long predates me - I'm just the messenger and you are
shooting from the hip.
Linus, the point I was making is that there are many, many ways to
control page cache invalidation and measure page cache residency,
and that trying to address them one-by-one is just a game of
whack-a-mole.
In future, can you please try not to go off the rails when someone
mentions O_DIRECT? You have a terrible habit of going off on
misdirected rants about O_DIRECT and/or XFS at any opportunity you
can get, and all it does is derail whatever useful conversation was
taking place.
Seriously. XFS is buggy in this regard, ext4 apparently isn't.
So you keep asserting despite being presented with evidence that it
mitigates other longstanding bugs that are really hard to solve.
Ignoring all the evidence you've been presented with and
re-asserting your original statement doesn't make it correct.
Did you not think to ask "what are those problems, and what can do
to solve them so we can remove the invalidation mitigations that XFS
uses?". That would be a useful contribution, whereas shouting about
how O_DIRECT is broken just pisses off the people working their
asses off to fix the problems you just heard about and are ranting
about.
Thinking that it's better to just invalidate the cache for direct IO
reads is all kinds of odd.
No, it's practicality. If we can't fix the problem, we have to
mitigate it. When we fix the underlying problem we can remove the
mitigation code. having you assert that it's broken and demand that
it be removed doesn't change the fact that we haven't fixed the
underlying problems. It's being worked on, but we're not there yet.
-Dave.
--
Dave Chinner
david@fromorbit.com
On Wed, Jan 9, 2019 at 11:04 PM Dave Chinner [off-list ref] wrote:
Sorry, what hacks did I just admit to making? This O_DIRECT
behaviour long predates me - I'm just the messenger and you are
shooting from the hip.
Sure, sorry. I find this whole thing annoying.
Linus, the point I was making is that there are many, many ways to
control page cache invalidation and measure page cache residency,
and that trying to address them one-by-one is just a game of
whack-a-mole.
.. and I agree. But let's a step back.Because there are different issues.
First off, the whole page cache attack is not necessarily something
many people will care about. As has been pointed out, it's often a
matter of convenience and (relative) portability.
And no, we're *never* going to stop all side channel leaks. Some parts
of caching (notably the timing effects of it) are pretty fundamental.
So at no point is this going to be some kind of absolute line in the
sand _anyway_. There is no black-and-white "you're protected", there's
only levels of convenience.
A remote attacker is hopefully going to be limited by the interfaces
to just timing attacks, although who knows what something like JS
might expose. Presumably neither mincore() nor arbitrary O_DIRECT or
pread2() flags.
Anyway, the reason I was trying to plug mincore() is largely that that
code didn't make much sense to begin with, and simply this:
mm/mincore.c | 94 +++++++++---------------------------------------------------
1 file changed, 13 insertions(+), 81 deletions(-)
if we can make people happier by removing lines of code and making the
semantics more clear anyway, it's worth trying.
No?
Is that everything? No. As mentioned, you'll never get to that "ok, we
plugged everything" point anyway. But removing a fairly easy way to
probe the cache that has no real upsides should be fairly
non-controversial.
But I do have to say that in many ways the page cache is *not* a great
attack vector because there's often lots of it, and it's fairly hard
to control. Once something is in the page cache for whatever reason,
it tends to be pretty sticky, and flushing it tends to be fairly hard
to predict.
And a cheap and residency (whether a simple probe like mincore of or a
NOWAIT flag) check is actually important just to try to control the
flushing part. Brute-forcing the flushing is generally very expensive,
but if you can't even see if you flushed it, it's way more so.
If there's a way to control the cache residency directly, that's
actually a much bigger hole than any residency check ever were.
Because once you can flush caches by reading, at that point you can
just flush a particular page and look at the IO stats for the root
partition or something. No residency check even needed.
So I do think that yes, as long as you can do a directed cache flush,
mincore is *entirely* immaterial.
Still, giving mincore clearer semantics and simpler code? Win-win.
(Except, of course, if somebody actually notices outside of tests.
Which may well happen and just force us to revert that commit. But
that's a separate issue entirely).
But I do think that we should strive to *never* invalidate caches on
read accesses. I don't actually see where you are doing that,
honestly: at least dio_complete() only does it for writes.
So I'm actually hoping that you are mis-remembering this and it turns
out that O_DIRECT reads don't invalidate caches.
Linus
(Except, of course, if somebody actually notices outside of tests.
Which may well happen and just force us to revert that commit. But
that's a separate issue entirely).
Both Dave and I pointed at a couple of utilities that break with
this. nocache can arguably work with the new behaviour but will behave
differently; vmtouch on the other hand is no longer able to display
what's in cache or not - people use that for example to "warm up" a
container in page cache based on how it appears after it had been
running for a while is a pretty valid usecase to me.
From the list Kevin harvested out of the debian code search, the
postgresql use case is pretty similar - probe what pages of the database
were in cache at shutdown so when you restart it you can preload these
and reach "cruse speed" faster.
Sure that's probably not billions of users but this all looks fairly
valid to me...
--
Dominique
On Thu, Jan 10, 2019 at 4:25 AM Dominique Martinet
[off-list ref] wrote:
Linus Torvalds wrote on Thu, Jan 10, 2019:
quoted
(Except, of course, if somebody actually notices outside of tests.
Which may well happen and just force us to revert that commit. But
that's a separate issue entirely).
Both Dave and I pointed at a couple of utilities that break with
this. nocache can arguably work with the new behaviour but will behave
differently; vmtouch on the other hand is no longer able to display
what's in cache or not - people use that for example to "warm up" a
container in page cache based on how it appears after it had been
running for a while is a pretty valid usecase to me.
So honestly, the main reason I'm loath to revert is that yes, we know
of theoretical differences, but they seem to all be
performance-related.
It would be really good to hear numbers. Is the warm-up optimization
something that changes things from 3ms to 3.5ms? Or does it change
things from 3ms to half a second?
Because Dave is absolutely correct that mincore() isn't really even
all that interesting an information leak if you can do the same with
RWF_NOWAIT. But the other side of that same coin is that if we're not
able to block mincore() sanely, then there's no point at looking at
RWF_NOWAIT either.
And we *can* do sane things about RWF_NOWAIT. For example, we could
start async IO on RWF_NOWAIT, and suddenly it would go from "probe the
page cache" to "probe and fill", and be much harder to use as an
attack vector..
Do we want to do that? Maybe, maybe not. But if mincore() can't be
fixed, there's no point in even trying.
Now, if the mincore() change results in a big performance hit for
people who use it as a heuristic for filling caches etc, then
reverting the trial balloon is obviously something we must do, but at
that point I'd also like to know which load it was that cared so much,
and just what it did. Because we did have an alternate patch that just
said "was the file writably opened, then we can do the page cache
probing". But at least one user (fincore) didn't do even that.
So right now, I consider the mincore change to be a "try to probe the
state of mincore users", and we haven't really gotten a lot of
information back yet.
Linus
From: Dave Chinner <david@fromorbit.com> Date: 2019-01-11 02:03:46
On Thu, Jan 10, 2019 at 02:11:01PM -0800, Linus Torvalds wrote:
And we *can* do sane things about RWF_NOWAIT. For example, we could
start async IO on RWF_NOWAIT, and suddenly it would go from "probe the
page cache" to "probe and fill", and be much harder to use as an
attack vector..
We can only do that if the application submits the read via AIO and
has an async IO completion reporting mechanism. Otherwise we have
to wait for the IO to complete to copy the data into the user's
buffer. And given that the app is using RWF_NOWAIT to explicitly
avoid blocking on the IO....
Also, keep in mind that RWF_NOWAIT also prevents blocking on
filesystem locks and full request queues. One of the prime drivers
of RWF_NOWAIT was to prevent AIO submission from blocking on
filesystem locks - it allows userspace to submit other IO while it
can't get all the access it requires to a single file or a single
block device is congested.
Hence I don't think there's a such a simple answer here - blocking
for IO breaks RWF_NOWAIT.
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
On Thu, Jan 10, 2019 at 6:03 PM Dave Chinner [off-list ref] wrote:
On Thu, Jan 10, 2019 at 02:11:01PM -0800, Linus Torvalds wrote:
quoted
And we *can* do sane things about RWF_NOWAIT. For example, we could
start async IO on RWF_NOWAIT, and suddenly it would go from "probe the
page cache" to "probe and fill", and be much harder to use as an
attack vector..
We can only do that if the application submits the read via AIO and
has an async IO completion reporting mechanism.
Oh, no, you misunderstand.
RWF_NOWAIT has a lot of situations where it will potentially return
early (the DAX and direct IO ones have their own), but I was thinking
of the one in generic_file_buffered_read(), which triggers when you
don't find a page mapping. That looks like the obvious "probe page
cache" case.
But we could literally move that test down just a few lines. Let it
start read-ahead.
.. and then it will actually trigger on the *second* case instead, where we have
if (!PageUptodate(page)) {
if (iocb->ki_flags & IOCB_NOWAIT) {
put_page(page);
goto would_block;
}
and that's where RWF_MNOWAIT would act.
It would still return EAGAIN.
But it would have started filling the page cache. So now the act of
probing would fill the page cache, and the attacker would be left high
and dry - the fact that the page cache now exists is because of the
attack, not because of whatever it was trying to measure.
See?
But obviously this kind of change only matters if we also have
mincore() not returning the probe data. mincore() obviously can't do
the same kind of read-ahead to defeat things.
Linus