On Wed, Aug 23, 2017 at 04:48:51PM -0700, Dan Williams wrote:

The mmap(2) syscall suffers from the ABI anti-pattern of not validating
unknown flags. However, proposals like MAP_SYNC and MAP_DIRECT need a
mechanism to define new behavior that is known to fail on older kernels
without the support. Define a new mmap3 syscall that checks for
unsupported flags at syscall entry and add a 'mmap_supported_mask' to
'struct file_operations' so generic code can validate the ->mmap()
handler knows about the specified flags. This also arranges for the
flags to be passed to the handler so it can do further local validation
if the requested behavior can be fulfilled.

What is the reason to not go with __MAP_VALID hack?  Adding new
syscalls is extremely painful, it will take forever to trickle this
through all architectures (especially with the various 32-bit
architectures having all kinds of different granularities for the
offset) and then the various C libraries, never mind applications.