From: David Herrmann <hidden> Date: 2014-06-17 10:01:55
Hi
On Tue, Jun 17, 2014 at 11:54 AM, Florian Weimer [off-list ref] wrote:
On 06/13/2014 05:33 PM, David Herrmann wrote:
quoted
On Fri, Jun 13, 2014 at 5:17 PM, Andy Lutomirski [off-list ref]
wrote:
quoted
Isn't the point of SEAL_SHRINK to allow servers to mmap and read
safely without worrying about SIGBUS?
No, I don't think so.
The point of SEAL_SHRINK is to prevent a file from shrinking. SIGBUS
is an effect, not a cause. It's only a coincidence that "OOM during
reads" and "reading beyond file-boundaries" has the same effect:
SIGBUS.
We only protect against reading beyond file-boundaries due to
shrinking. Therefore, OOM-SIGBUS is unrelated to SEAL_SHRINK.
Anyone dealing with mmap() _has_ to use mlock() to protect against
OOM-SIGBUS. Making SEAL_SHRINK protect against OOM-SIGBUS would be
redundant, because you can achieve the same with SEAL_SHRINK+mlock().
I don't think this is what potential users expect because mlock requires
capabilities which are not available to them.
A couple of weeks ago, sealing was to be applied to anonymous shared memory.
Has this changed? Why should *reading* it trigger OOM?
The file might have holes, therefore, you'd have to allocate backing
pages. This might hit a soft-limit and fail. To avoid this, use
fallocate() to allocate pages prior to mmap() or mlock() to make the
kernel lock them in memory.
Thanks
David
I don't think this is what potential users expect because mlock requires
capabilities which are not available to them.
A couple of weeks ago, sealing was to be applied to anonymous shared memory.
Has this changed? Why should *reading* it trigger OOM?
The file might have holes, therefore, you'd have to allocate backing
pages. This might hit a soft-limit and fail. To avoid this, use
fallocate() to allocate pages prior to mmap()
This does not work because the consuming side does not know how the
descriptor was set up if sealing does not imply that.
or mlock() to make the kernel lock them in memory.
See above for why that does not work.
I think you should eliminate the holes on sealing and report ENOMEM
there if necessary.
--
Florian Weimer / Red Hat Product Security Team
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
From: David Herrmann <hidden> Date: 2014-06-17 10:10:20
Hi
On Tue, Jun 17, 2014 at 12:04 PM, Florian Weimer [off-list ref] wrote:
On 06/17/2014 12:01 PM, David Herrmann wrote:
quoted
quoted
I don't think this is what potential users expect because mlock requires
capabilities which are not available to them.
A couple of weeks ago, sealing was to be applied to anonymous shared
memory.
Has this changed? Why should *reading* it trigger OOM?
The file might have holes, therefore, you'd have to allocate backing
pages. This might hit a soft-limit and fail. To avoid this, use
fallocate() to allocate pages prior to mmap()
This does not work because the consuming side does not know how the
descriptor was set up if sealing does not imply that.
The consuming side has to very seals via F_GET_SEALS. After that, it
shall do a simple fallocate() on the whole file if it wants to go sure
that all pages are allocated. Why shouldn't that be possible? Please
elaborate.
Thanks
David
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
The file might have holes, therefore, you'd have to allocate backing
pages. This might hit a soft-limit and fail. To avoid this, use
fallocate() to allocate pages prior to mmap()
This does not work because the consuming side does not know how the
descriptor was set up if sealing does not imply that.
The consuming side has to very seals via F_GET_SEALS. After that, it
shall do a simple fallocate() on the whole file if it wants to go sure
that all pages are allocated. Why shouldn't that be possible? Please
elaborate.
Hmm. You permit general fallocate even for WRITE seals. That's really
unexpected.
The inode_newsize_ok check in shmem_fallocate can result in SIGXFSZ,
which doesn't seem to be what's intended here.
Will the new pages attributed to the process calling fallocate, or to
the process calling memfd_create?
--
Florian Weimer / Red Hat Product Security Team
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
From: Andy Lutomirski <luto@amacapital.net> Date: 2014-06-17 16:20:46
On Jun 17, 2014 3:01 AM, "David Herrmann" [off-list ref] wrote:
Hi
On Tue, Jun 17, 2014 at 11:54 AM, Florian Weimer [off-list ref]
wrote:
quoted
On 06/13/2014 05:33 PM, David Herrmann wrote:
quoted
On Fri, Jun 13, 2014 at 5:17 PM, Andy Lutomirski [off-list ref]
wrote:
quoted
Isn't the point of SEAL_SHRINK to allow servers to mmap and read
safely without worrying about SIGBUS?
No, I don't think so.
The point of SEAL_SHRINK is to prevent a file from shrinking. SIGBUS
is an effect, not a cause. It's only a coincidence that "OOM during
reads" and "reading beyond file-boundaries" has the same effect:
SIGBUS.
We only protect against reading beyond file-boundaries due to
shrinking. Therefore, OOM-SIGBUS is unrelated to SEAL_SHRINK.
Anyone dealing with mmap() _has_ to use mlock() to protect against
OOM-SIGBUS. Making SEAL_SHRINK protect against OOM-SIGBUS would be
redundant, because you can achieve the same with SEAL_SHRINK+mlock().
I don't think this is what potential users expect because mlock requires
capabilities which are not available to them.
A couple of weeks ago, sealing was to be applied to anonymous shared
memory.
quoted
Has this changed? Why should *reading* it trigger OOM?
The file might have holes, therefore, you'd have to allocate backing
pages. This might hit a soft-limit and fail. To avoid this, use
fallocate() to allocate pages prior to mmap() or mlock() to make the
kernel lock them in memory.
Can you summarize why holes can't be reliably backed by the zero page?
(I realize the kernel could OOM on PTE allocation, but fallocate won't fix
that. OTOH MAP_POPULATE should work.)
And I don't think I like hole filling being allowed on write-sealed files.
Holes are observable these days with SEEK_HOLE and such.
Alternatively, we could add a new syscall or madvise option to populate a
mapping.
--Andy
From: David Herrmann <hidden> Date: 2014-06-17 16:36:55
Hi
On Tue, Jun 17, 2014 at 6:20 PM, Andy Lutomirski [off-list ref] wrote:
Can you summarize why holes can't be reliably backed by the zero page?
To answer this, I will quote Hugh from "PATCH v2 1/3":
We do already use the ZERO_PAGE instead of allocating when it's a
simple read; and on the face of it, we could extend that to mmap
once the file is sealed. But I am rather afraid to do so - for
many years there was an mmap /dev/zero case which did that, but
it was an easily forgotten case which caught us out at least
once, so I'm reluctant to reintroduce it now for sealing.
Anyway, I don't expect you to resolve the issue of sealed holes:
that's very much my territory, to give you support on.
Holes can be avoided with a simple fallocate(). I don't understand why
I should make SEAL_WRITE do the fallocate for the caller. During the
discussion of memfd_create() I was told to drop the "size" parameter,
because it is redundant. I don't see how this implicit fallocate()
does not fall into the same category?
Thanks
David
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
From: Andy Lutomirski <luto@amacapital.net> Date: 2014-06-17 16:41:22
On Tue, Jun 17, 2014 at 9:36 AM, David Herrmann [off-list ref] wrote:
Hi
On Tue, Jun 17, 2014 at 6:20 PM, Andy Lutomirski [off-list ref] wrote:
quoted
Can you summarize why holes can't be reliably backed by the zero page?
To answer this, I will quote Hugh from "PATCH v2 1/3":
quoted
We do already use the ZERO_PAGE instead of allocating when it's a
simple read; and on the face of it, we could extend that to mmap
once the file is sealed. But I am rather afraid to do so - for
many years there was an mmap /dev/zero case which did that, but
it was an easily forgotten case which caught us out at least
once, so I'm reluctant to reintroduce it now for sealing.
Anyway, I don't expect you to resolve the issue of sealed holes:
that's very much my territory, to give you support on.
Holes can be avoided with a simple fallocate(). I don't understand why
I should make SEAL_WRITE do the fallocate for the caller. During the
discussion of memfd_create() I was told to drop the "size" parameter,
because it is redundant. I don't see how this implicit fallocate()
does not fall into the same category?
I'm really confused now.
If I SEAL_WRITE a file, and then I mmap it PROT_READ, and then I read
it, is that a "simple read"? If so, doesn't that mean that there's no
problem?
--Andy
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
From: David Herrmann <hidden> Date: 2014-06-17 16:51:18
Hi
On Tue, Jun 17, 2014 at 6:41 PM, Andy Lutomirski [off-list ref] wrote:
On Tue, Jun 17, 2014 at 9:36 AM, David Herrmann [off-list ref] wrote:
quoted
Hi
On Tue, Jun 17, 2014 at 6:20 PM, Andy Lutomirski [off-list ref] wrote:
quoted
Can you summarize why holes can't be reliably backed by the zero page?
To answer this, I will quote Hugh from "PATCH v2 1/3":
quoted
We do already use the ZERO_PAGE instead of allocating when it's a
simple read; and on the face of it, we could extend that to mmap
once the file is sealed. But I am rather afraid to do so - for
many years there was an mmap /dev/zero case which did that, but
it was an easily forgotten case which caught us out at least
once, so I'm reluctant to reintroduce it now for sealing.
Anyway, I don't expect you to resolve the issue of sealed holes:
that's very much my territory, to give you support on.
Holes can be avoided with a simple fallocate(). I don't understand why
I should make SEAL_WRITE do the fallocate for the caller. During the
discussion of memfd_create() I was told to drop the "size" parameter,
because it is redundant. I don't see how this implicit fallocate()
does not fall into the same category?
I'm really confused now.
If I SEAL_WRITE a file, and then I mmap it PROT_READ, and then I read
it, is that a "simple read"? If so, doesn't that mean that there's no
problem?
I assumed Hugh was talking about read(). So no, this is not about
memory-reads on mmap()ed regions.
Looking at shmem_file_read_iter() I can see a ZERO_PAGE(0) call in
case shmem_getpage_gfp(SGP_READ) tells us there's a hole. I cannot see
anything like that in the mmap_region() and shmem_fault() paths.
Thanks
David
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
From: Andy Lutomirski <luto@amacapital.net> Date: 2014-06-17 17:01:05
On Tue, Jun 17, 2014 at 9:51 AM, David Herrmann [off-list ref] wrote:
Hi
On Tue, Jun 17, 2014 at 6:41 PM, Andy Lutomirski [off-list ref] wrote:
quoted
On Tue, Jun 17, 2014 at 9:36 AM, David Herrmann [off-list ref] wrote:
quoted
Hi
On Tue, Jun 17, 2014 at 6:20 PM, Andy Lutomirski [off-list ref] wrote:
quoted
Can you summarize why holes can't be reliably backed by the zero page?
To answer this, I will quote Hugh from "PATCH v2 1/3":
quoted
We do already use the ZERO_PAGE instead of allocating when it's a
simple read; and on the face of it, we could extend that to mmap
once the file is sealed. But I am rather afraid to do so - for
many years there was an mmap /dev/zero case which did that, but
it was an easily forgotten case which caught us out at least
once, so I'm reluctant to reintroduce it now for sealing.
Anyway, I don't expect you to resolve the issue of sealed holes:
that's very much my territory, to give you support on.
Holes can be avoided with a simple fallocate(). I don't understand why
I should make SEAL_WRITE do the fallocate for the caller. During the
discussion of memfd_create() I was told to drop the "size" parameter,
because it is redundant. I don't see how this implicit fallocate()
does not fall into the same category?
I'm really confused now.
If I SEAL_WRITE a file, and then I mmap it PROT_READ, and then I read
it, is that a "simple read"? If so, doesn't that mean that there's no
problem?
I assumed Hugh was talking about read(). So no, this is not about
memory-reads on mmap()ed regions.
Looking at shmem_file_read_iter() I can see a ZERO_PAGE(0) call in
case shmem_getpage_gfp(SGP_READ) tells us there's a hole. I cannot see
anything like that in the mmap_region() and shmem_fault() paths.
Would it be easy to fix this just for SEAL_WRITE files? Hugh?
This would make the interface much nicer, IMO.
--Andy
Thanks
David
--
Andy Lutomirski
AMA Capital Management, LLC
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>