From: Will Deacon <will@kernel.org> Date: 2021-07-30 11:24:57
Hi everyone,
This is version eleven of the patches previously posted here:
v1: https://lore.kernel.org/r/20201027215118.27003-1-will@kernel.org
v2: https://lore.kernel.org/r/20201109213023.15092-1-will@kernel.org
v3: https://lore.kernel.org/r/20201113093720.21106-1-will@kernel.org
v4: https://lore.kernel.org/r/20201124155039.13804-1-will@kernel.org
v5: https://lore.kernel.org/r/20201208132835.6151-1-will@kernel.org
v6: https://lore.kernel.org/r/20210518094725.7701-1-will@kernel.org
v7: https://lore.kernel.org/r/20210525151432.16875-1-will@kernel.org
v8: https://lore.kernel.org/r/20210602164719.31777-1-will@kernel.org
v9: https://lore.kernel.org/r/20210608180313.11502-1-will@kernel.org
v10: https://lore.kernel.org/r/20210623173848.318-1-will@kernel.org
The main changes since v10 are:
* Now based on v5.14-rc1
* Fixed a lockup found during testing where select_fallback_rq()
could return an invalid CPU when trying to prioritise a destination
on the same NUMA node
Greg kindly pointed out a potential issue with the new 'aarch32_el0'
file in sysfs, where the contents will be truncated if we need to
display more than a PAGE_SIZE of data. However, I have elected not to
address this for now as (a) I do not think we'll ever have hardware in
that configuration (yeah, I know...) and (b) It is useful for scripts
if the file behaves the same as the other files in the same directory
(e.g. 'online'). Should a solution to this problem emerge for the other
files, I will be happy to adopt it here as well.
This series is now mostly scheduler stuff, but the last few patches are
arm64 and so I would suggest a shared branch in -tip for merging.
Cheers,
Will
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Marc Zyngier <maz@kernel.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Morten Rasmussen <redacted>
Cc: Qais Yousef <redacted>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Quentin Perret <redacted>
Cc: Tejun Heo <tj@kernel.org>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Juri Lelli <juri.lelli@redhat.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Cc: "Rafael J. Wysocki" <redacted>
Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
Cc: Daniel Bristot de Oliveira <redacted>
Cc: Valentin Schneider <redacted>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: kernel-team@android.com
--->8
Will Deacon (16):
sched: Introduce task_cpu_possible_mask() to limit fallback rq
selection
cpuset: Don't use the cpu_possible_mask as a last resort for cgroup v1
cpuset: Honour task_cpu_possible_mask() in guarantee_online_cpus()
cpuset: Cleanup cpuset_cpus_allowed_fallback() use in
select_fallback_rq()
sched: Reject CPU affinity changes based on task_cpu_possible_mask()
sched: Introduce task_struct::user_cpus_ptr to track requested
affinity
sched: Split the guts of sched_setaffinity() into a helper function
sched: Allow task CPU affinity to be restricted on asymmetric systems
sched: Introduce dl_task_check_affinity() to check proposed affinity
arm64: Implement task_cpu_possible_mask()
arm64: exec: Adjust affinity for compat tasks with mismatched 32-bit
EL0
arm64: Prevent offlining first CPU with 32-bit EL0 on mismatched
system
arm64: Advertise CPUs capable of running 32-bit applications in sysfs
arm64: Hook up cmdline parameter to allow mismatched 32-bit EL0
arm64: Remove logic to kill 32-bit tasks on 64-bit-only cores
Documentation: arm64: describe asymmetric 32-bit support
.../ABI/testing/sysfs-devices-system-cpu | 9 +
.../admin-guide/kernel-parameters.txt | 11 +
Documentation/arm64/asymmetric-32bit.rst | 155 ++++++++
Documentation/arm64/index.rst | 1 +
arch/arm64/include/asm/elf.h | 6 +-
arch/arm64/include/asm/mmu_context.h | 13 +
arch/arm64/kernel/cpufeature.c | 51 ++-
arch/arm64/kernel/process.c | 47 ++-
arch/arm64/kernel/signal.c | 26 --
include/linux/cpuset.h | 8 +-
include/linux/mmu_context.h | 14 +
include/linux/sched.h | 21 ++
init/init_task.c | 1 +
kernel/cgroup/cpuset.c | 59 ++-
kernel/fork.c | 2 +
kernel/sched/core.c | 344 ++++++++++++++----
kernel/sched/sched.h | 1 +
17 files changed, 628 insertions(+), 141 deletions(-)
create mode 100644 Documentation/arm64/asymmetric-32bit.rst
--
2.32.0.402.g57bb445576-goog
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Will Deacon <will@kernel.org> Date: 2021-07-30 11:24:58
Asymmetric systems may not offer the same level of userspace ISA support
across all CPUs, meaning that some applications cannot be executed by
some CPUs. As a concrete example, upcoming arm64 big.LITTLE designs do
not feature support for 32-bit applications on both clusters.
On such a system, we must take care not to migrate a task to an
unsupported CPU when forcefully moving tasks in select_fallback_rq()
in response to a CPU hot-unplug operation.
Introduce a task_cpu_possible_mask() hook which, given a task argument,
allows an architecture to return a cpumask of CPUs that are capable of
executing that task. The default implementation returns the
cpu_possible_mask, since sane machines do not suffer from per-cpu ISA
limitations that affect scheduling. The new mask is used when selecting
the fallback runqueue as a last resort before forcing a migration to the
first active CPU.
Reviewed-by: Valentin Schneider <redacted>
Reviewed-by: Quentin Perret <redacted>
Signed-off-by: Will Deacon <will@kernel.org>
---
include/linux/mmu_context.h | 14 ++++++++++++++
kernel/sched/core.c | 9 +++------
2 files changed, 17 insertions(+), 6 deletions(-)
@@ -2163,7 +2163,7 @@ static inline bool is_cpu_allowed(struct task_struct *p, int cpu)/* Non kernel threads are not allowed during either online or offline. */if(!(p->flags&PF_KTHREAD))-returncpu_active(cpu);+returncpu_active(cpu)&&task_cpu_possible(cpu,p);/* KTHREAD_IS_PER_CPU is always allowed. */if(kthread_is_per_cpu(p))
@@ -3114,9 +3114,7 @@ static int select_fallback_rq(int cpu, struct task_struct *p)/* Look for allowed, online CPU in same node. */for_each_cpu(dest_cpu,nodemask){-if(!cpu_active(dest_cpu))-continue;-if(cpumask_test_cpu(dest_cpu,p->cpus_ptr))+if(is_cpu_allowed(p,dest_cpu))returndest_cpu;}}
From: Will Deacon <will@kernel.org> Date: 2021-07-30 11:25:03
If the scheduler cannot find an allowed CPU for a task,
cpuset_cpus_allowed_fallback() will widen the affinity to cpu_possible_mask
if cgroup v1 is in use.
In preparation for allowing architectures to provide their own fallback
mask, just return early if we're either using cgroup v1 or we're using
cgroup v2 with a mask that contains invalid CPUs. This will allow
select_fallback_rq() to figure out the mask by itself.
Cc: Tejun Heo <tj@kernel.org>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Reviewed-by: Valentin Schneider <redacted>
Reviewed-by: Quentin Perret <redacted>
Signed-off-by: Will Deacon <will@kernel.org>
---
include/linux/cpuset.h | 1 +
kernel/cgroup/cpuset.c | 8 ++++++--
2 files changed, 7 insertions(+), 2 deletions(-)
From: Will Deacon <will@kernel.org> Date: 2021-07-30 11:25:08
Asymmetric systems may not offer the same level of userspace ISA support
across all CPUs, meaning that some applications cannot be executed by
some CPUs. As a concrete example, upcoming arm64 big.LITTLE designs do
not feature support for 32-bit applications on both clusters.
Modify guarantee_online_cpus() to take task_cpu_possible_mask() into
account when trying to find a suitable set of online CPUs for a given
task. This will avoid passing an invalid mask to set_cpus_allowed_ptr()
during ->attach() and will subsequently allow the cpuset hierarchy to be
taken into account when forcefully overriding the affinity mask for a
task which requires migration to a compatible CPU.
Cc: Tejun Heo <tj@kernel.org>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Reviewed-by: Valentin Schneider <redacted>
Signed-off-by: Will Deacon <will@kernel.org>
---
include/linux/cpuset.h | 2 +-
kernel/cgroup/cpuset.c | 43 +++++++++++++++++++++++++-----------------
2 files changed, 27 insertions(+), 18 deletions(-)
From: Will Deacon <will@kernel.org> Date: 2021-07-30 11:25:13
select_fallback_rq() only needs to recheck for an allowed CPU if the
affinity mask of the task has changed since the last check.
Return a 'bool' from cpuset_cpus_allowed_fallback() to indicate whether
the affinity mask was updated, and use this to elide the allowed check
when the mask has been left alone.
No functional change.
Suggested-by: Valentin Schneider <redacted>
Reviewed-by: Valentin Schneider <redacted>
Signed-off-by: Will Deacon <will@kernel.org>
---
include/linux/cpuset.h | 5 +++--
kernel/cgroup/cpuset.c | 10 ++++++++--
kernel/sched/core.c | 3 +--
3 files changed, 12 insertions(+), 6 deletions(-)
@@ -3131,8 +3131,7 @@ static int select_fallback_rq(int cpu, struct task_struct *p)/* No more Mr. Nice Guy. */switch(state){casecpuset:-if(IS_ENABLED(CONFIG_CPUSETS)){-cpuset_cpus_allowed_fallback(p);+if(cpuset_cpus_allowed_fallback(p)){state=possible;break;}
--
2.32.0.402.g57bb445576-goog
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Will Deacon <will@kernel.org> Date: 2021-07-30 11:25:19
Reject explicit requests to change the affinity mask of a task via
set_cpus_allowed_ptr() if the requested mask is not a subset of the
mask returned by task_cpu_possible_mask(). This ensures that the
'cpus_mask' for a given task cannot contain CPUs which are incapable of
executing it, except in cases where the affinity is forced.
Reviewed-by: Valentin Schneider <redacted>
Reviewed-by: Quentin Perret <redacted>
Signed-off-by: Will Deacon <will@kernel.org>
---
kernel/sched/core.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
From: Will Deacon <will@kernel.org> Date: 2021-07-30 11:25:21
In preparation for saving and restoring the user-requested CPU affinity
mask of a task, add a new cpumask_t pointer to 'struct task_struct'.
If the pointer is non-NULL, then the mask is copied across fork() and
freed on task exit.
Reviewed-by: Valentin Schneider <redacted>
Signed-off-by: Will Deacon <will@kernel.org>
---
include/linux/sched.h | 13 +++++++++++++
init/init_task.c | 1 +
kernel/fork.c | 2 ++
kernel/sched/core.c | 20 ++++++++++++++++++++
4 files changed, 36 insertions(+)
From: Will Deacon <will@kernel.org> Date: 2021-07-30 11:25:28
In preparation for replaying user affinity requests using a saved mask,
split sched_setaffinity() up so that the initial task lookup and
security checks are only performed when the request is coming directly
from userspace.
Reviewed-by: Valentin Schneider <redacted>
Signed-off-by: Will Deacon <will@kernel.org>
---
kernel/sched/core.c | 105 ++++++++++++++++++++++++--------------------
1 file changed, 57 insertions(+), 48 deletions(-)
From: Peter Zijlstra <peterz@infradead.org> Date: 2021-08-17 15:41:49
On Fri, Jul 30, 2021 at 12:24:34PM +0100, Will Deacon wrote:
In preparation for replaying user affinity requests using a saved mask,
split sched_setaffinity() up so that the initial task lookup and
security checks are only performed when the request is coming directly
from userspace.
Reviewed-by: Valentin Schneider <redacted>
Signed-off-by: Will Deacon <will@kernel.org>
Should not sched_setaffinity() update user_cpus_ptr when it isn't NULL,
such that the upcoming relax_compatible_cpus_allowed_ptr() preserve the
full user mask?
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Will Deacon <will@kernel.org> Date: 2021-08-18 10:50:48
On Tue, Aug 17, 2021 at 05:40:24PM +0200, Peter Zijlstra wrote:
On Fri, Jul 30, 2021 at 12:24:34PM +0100, Will Deacon wrote:
quoted
In preparation for replaying user affinity requests using a saved mask,
split sched_setaffinity() up so that the initial task lookup and
security checks are only performed when the request is coming directly
from userspace.
Reviewed-by: Valentin Schneider <redacted>
Signed-off-by: Will Deacon <will@kernel.org>
Should not sched_setaffinity() update user_cpus_ptr when it isn't NULL,
such that the upcoming relax_compatible_cpus_allowed_ptr() preserve the
full user mask?
The idea is that force_compatible_cpus_allowed_ptr() and
relax_compatible_cpus_allowed_ptr() are used as a pair, with the former
setting ->user_cpus_ptr and the latter restoring it. An intervening call
to sched_setaffinity() must _clear_ the saved mask, as we discussed
before at:
https://lore.kernel.org/r/YK53kDtczHIYumDC@hirez.programming.kicks-ass.net
Will
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Peter Zijlstra <peterz@infradead.org> Date: 2021-08-18 10:59:08
On Wed, Aug 18, 2021 at 11:50:30AM +0100, Will Deacon wrote:
On Tue, Aug 17, 2021 at 05:40:24PM +0200, Peter Zijlstra wrote:
quoted
On Fri, Jul 30, 2021 at 12:24:34PM +0100, Will Deacon wrote:
quoted
In preparation for replaying user affinity requests using a saved mask,
split sched_setaffinity() up so that the initial task lookup and
security checks are only performed when the request is coming directly
from userspace.
Reviewed-by: Valentin Schneider <redacted>
Signed-off-by: Will Deacon <will@kernel.org>
Should not sched_setaffinity() update user_cpus_ptr when it isn't NULL,
such that the upcoming relax_compatible_cpus_allowed_ptr() preserve the
full user mask?
The idea is that force_compatible_cpus_allowed_ptr() and
relax_compatible_cpus_allowed_ptr() are used as a pair, with the former
setting ->user_cpus_ptr and the latter restoring it. An intervening call
to sched_setaffinity() must _clear_ the saved mask, as we discussed
before at:
https://lore.kernel.org/r/YK53kDtczHIYumDC@hirez.programming.kicks-ass.net
Clearly that deserves a comment somewhere, because I keep trying to make
it more consistent than it can be :/ I'll see if I can find a spot.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Will Deacon <will@kernel.org> Date: 2021-08-18 11:13:06
On Wed, Aug 18, 2021 at 12:56:24PM +0200, Peter Zijlstra wrote:
On Wed, Aug 18, 2021 at 11:50:30AM +0100, Will Deacon wrote:
quoted
On Tue, Aug 17, 2021 at 05:40:24PM +0200, Peter Zijlstra wrote:
quoted
On Fri, Jul 30, 2021 at 12:24:34PM +0100, Will Deacon wrote:
quoted
In preparation for replaying user affinity requests using a saved mask,
split sched_setaffinity() up so that the initial task lookup and
security checks are only performed when the request is coming directly
from userspace.
Reviewed-by: Valentin Schneider <redacted>
Signed-off-by: Will Deacon <will@kernel.org>
Should not sched_setaffinity() update user_cpus_ptr when it isn't NULL,
such that the upcoming relax_compatible_cpus_allowed_ptr() preserve the
full user mask?
The idea is that force_compatible_cpus_allowed_ptr() and
relax_compatible_cpus_allowed_ptr() are used as a pair, with the former
setting ->user_cpus_ptr and the latter restoring it. An intervening call
to sched_setaffinity() must _clear_ the saved mask, as we discussed
before at:
https://lore.kernel.org/r/YK53kDtczHIYumDC@hirez.programming.kicks-ass.net
Clearly that deserves a comment somewhere, because I keep trying to make
it more consistent than it can be :/ I'll see if I can find a spot.
Agreed. The relax/force functions are already commented, so maybe alongside
SCA_USER?
Will
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Will Deacon <will@kernel.org> Date: 2021-07-30 11:25:31
Asymmetric systems may not offer the same level of userspace ISA support
across all CPUs, meaning that some applications cannot be executed by
some CPUs. As a concrete example, upcoming arm64 big.LITTLE designs do
not feature support for 32-bit applications on both clusters.
Although userspace can carefully manage the affinity masks for such
tasks, one place where it is particularly problematic is execve()
because the CPU on which the execve() is occurring may be incompatible
with the new application image. In such a situation, it is desirable to
restrict the affinity mask of the task and ensure that the new image is
entered on a compatible CPU. From userspace's point of view, this looks
the same as if the incompatible CPUs have been hotplugged off in the
task's affinity mask. Similarly, if a subsequent execve() reverts to
a compatible image, then the old affinity is restored if it is still
valid.
In preparation for restricting the affinity mask for compat tasks on
arm64 systems without uniform support for 32-bit applications, introduce
{force,relax}_compatible_cpus_allowed_ptr(), which respectively restrict
and restore the affinity mask for a task based on the compatible CPUs.
Reviewed-by: Valentin Schneider <redacted>
Reviewed-by: Quentin Perret <redacted>
Signed-off-by: Will Deacon <will@kernel.org>
---
include/linux/sched.h | 2 +
kernel/sched/core.c | 180 ++++++++++++++++++++++++++++++++++++++----
kernel/sched/sched.h | 1 +
3 files changed, 167 insertions(+), 16 deletions(-)
@@ -2783,20 +2778,173 @@ static int __set_cpus_allowed_ptr(struct task_struct *p,__do_set_cpus_allowed(p,new_mask,flags);-returnaffine_move_task(rq,p,&rf,dest_cpu,flags);+if(flags&SCA_USER)+release_user_cpus_ptr(p);++returnaffine_move_task(rq,p,rf,dest_cpu,flags);out:-task_rq_unlock(rq,p,&rf);+task_rq_unlock(rq,p,rf);returnret;}+/*+*Changeagiventask'sCPUaffinity.Migratethethreadtoa+*properCPUandscheduleitawayiftheCPUit'sexecutingon+*isremovedfromtheallowedbitmask.+*+*NOTE:thecallermusthaveavalidreferencetothetask,the+*taskmustnotexit()&deallocateitselfprematurely.The+*callisnotatomic;nospinlocksmaybeheld.+*/+staticint__set_cpus_allowed_ptr(structtask_struct*p,+conststructcpumask*new_mask,u32flags)+{+structrq_flagsrf;+structrq*rq;++rq=task_rq_lock(p,&rf);+return__set_cpus_allowed_ptr_locked(p,new_mask,flags,rq,&rf);+}+intset_cpus_allowed_ptr(structtask_struct*p,conststructcpumask*new_mask){return__set_cpus_allowed_ptr(p,new_mask,0);}EXPORT_SYMBOL_GPL(set_cpus_allowed_ptr);+/*+*Changeagiventask'sCPUaffinitytotheintersectionofitscurrent+*affinitymaskand@subset_mask,writingtheresultingmaskto@new_mask+*andpointing@p->user_cpus_ptrtoacopyoftheoldmask.+*Iftheresultingmaskisempty,leavetheaffinityunchangedandreturn+*-EINVAL.+*/+staticintrestrict_cpus_allowed_ptr(structtask_struct*p,+structcpumask*new_mask,+conststructcpumask*subset_mask)+{+structrq_flagsrf;+structrq*rq;+interr;+structcpumask*user_mask=NULL;++if(!p->user_cpus_ptr){+user_mask=kmalloc(cpumask_size(),GFP_KERNEL);++if(!user_mask)+return-ENOMEM;+}++rq=task_rq_lock(p,&rf);++/*+*Forcefullyrestrictingtheaffinityofadeadlinetaskis+*likelytocauseproblems,sofailandnoisilyoverridethe+*maskentirely.+*/+if(task_has_dl_policy(p)&&dl_bandwidth_enabled()){+err=-EPERM;+gotoerr_unlock;+}++if(!cpumask_and(new_mask,&p->cpus_mask,subset_mask)){+err=-EINVAL;+gotoerr_unlock;+}++/*+*We'reabouttobutcherthetaskaffinity,sokeeptrackofwhat+*theuseraskedforincasewe'reabletorestoreitlateron.+*/+if(user_mask){+cpumask_copy(user_mask,p->cpus_ptr);+p->user_cpus_ptr=user_mask;+}++return__set_cpus_allowed_ptr_locked(p,new_mask,0,rq,&rf);++err_unlock:+task_rq_unlock(rq,p,&rf);+kfree(user_mask);+returnerr;+}++/*+*RestricttheCPUaffinityoftask@psothatitisasubsetof+*task_cpu_possible_mask()andpoint@p->user_cpu_ptrtoacopyofthe+*oldaffinitymask.Iftheresultingmaskisempty,wewarnandwalk+*upthecpusethierarchyuntilwefindasuitablemask.+*/+voidforce_compatible_cpus_allowed_ptr(structtask_struct*p)+{+cpumask_var_tnew_mask;+conststructcpumask*override_mask=task_cpu_possible_mask(p);++alloc_cpumask_var(&new_mask,GFP_KERNEL);++/*+*__migrate_task()canfailsilentlyinthefaceofconcurrent+*offliningofthechosendestinationCPU,sotakethehotplug+*locktoensurethatthemigrationsucceeds.+*/+cpus_read_lock();+if(!cpumask_available(new_mask))+gotoout_set_mask;++if(!restrict_cpus_allowed_ptr(p,new_mask,override_mask))+gotoout_free_mask;++/*+*Wefailedtofindavalidsubsetoftheaffinitymaskforthe+*task,sooverrideitbasedonitscpusethierarchy.+*/+cpuset_cpus_allowed(p,new_mask);+override_mask=new_mask;++out_set_mask:+if(printk_ratelimit()){+printk_deferred("Overriding affinity for process %d (%s) to CPUs %*pbl\n",+task_pid_nr(p),p->comm,+cpumask_pr_args(override_mask));+}++WARN_ON(set_cpus_allowed_ptr(p,override_mask));+out_free_mask:+cpus_read_unlock();+free_cpumask_var(new_mask);+}++staticint+__sched_setaffinity(structtask_struct*p,conststructcpumask*mask);++/*+*Restoretheaffinityofatask@pwhichwaspreviouslyrestrictedbya+*calltoforce_compatible_cpus_allowed_ptr().Thiswillclear(andfree)+*@p->user_cpus_ptr.+*+*Itisthecaller'sresponsibilitytoserialisethiswithanycallsto+*force_compatible_cpus_allowed_ptr(@p).+*/+voidrelax_compatible_cpus_allowed_ptr(structtask_struct*p)+{+unsignedlongflags;+structcpumask*mask=p->user_cpus_ptr;++/*+*Trytorestoretheoldaffinitymask.Ifthisfails,then+*wefreethemaskexplicitlytoavoiditbeinginheritedacross+*asubsequentfork().+*/+if(!mask||!__sched_setaffinity(p,mask))+return;++raw_spin_lock_irqsave(&p->pi_lock,flags);+release_user_cpus_ptr(p);+raw_spin_unlock_irqrestore(&p->pi_lock,flags);+}+voidset_task_cpu(structtask_struct*p,unsignedintnew_cpu){#ifdef CONFIG_SCHED_DEBUG
+void relax_compatible_cpus_allowed_ptr(struct task_struct *p)
+{
+ unsigned long flags;
+ struct cpumask *mask = p->user_cpus_ptr;
+
+ /*
+ * Try to restore the old affinity mask. If this fails, then
+ * we free the mask explicitly to avoid it being inherited across
+ * a subsequent fork().
+ */
+ if (!mask || !__sched_setaffinity(p, mask))
+ return;
+
+ raw_spin_lock_irqsave(&p->pi_lock, flags);
+ release_user_cpus_ptr(p);
+ raw_spin_unlock_irqrestore(&p->pi_lock, flags);
+}
Both these are a problem on RT.
The easiest recourse is simply never freeing the CPU mask (except on
exit). The alternative is something like the below I suppose..
I'm leaning towards the former option, wdyt?
+void relax_compatible_cpus_allowed_ptr(struct task_struct *p)
+{
+ unsigned long flags;
+ struct cpumask *mask = p->user_cpus_ptr;
+
+ /*
+ * Try to restore the old affinity mask. If this fails, then
+ * we free the mask explicitly to avoid it being inherited across
+ * a subsequent fork().
+ */
+ if (!mask || !__sched_setaffinity(p, mask))
+ return;
+
+ raw_spin_lock_irqsave(&p->pi_lock, flags);
+ release_user_cpus_ptr(p);
+ raw_spin_unlock_irqrestore(&p->pi_lock, flags);
+}
Both these are a problem on RT.
Ah, sorry. I didn't realise you couldn't _free_ with a raw lock held in RT.
Is there somewhere I can read up on that?
The easiest recourse is simply never freeing the CPU mask (except on
exit). The alternative is something like the below I suppose..
I'm leaning towards the former option, wdyt?
Defering the freeing until exit feels like a little fiddly, as we still
want to clear ->user_cpus_ptr on affinity changes when SCA_USER is set
so we'd have to keep track of the mask somewhere and reuse it instead
of allocating a new one if we need it later on. Do-able, but feels a bit
nasty, particular across fork().
As for your other suggestion:
I think the idea looks good, but perhaps we could wrap things up a bit:
/* Comment about why this is useful with RT */
static cpumask_t *clear_user_cpus_ptr(struct task_struct *p)
{
struct cpumask *user_mask = NULL;
swap(user_mask, p->user_cpus_ptr);
return user_mask;
}
void release_user_cpus_ptr(struct task_struct *p)
{
kfree(clear_user_cpus_ptr(p));
}
Then just use clear_user_cpus_ptr() in sched/core.c where we know what
we're doing (well, at least one of us does!).
Will
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
I think the idea looks good, but perhaps we could wrap things up a bit:
/* Comment about why this is useful with RT */
static cpumask_t *clear_user_cpus_ptr(struct task_struct *p)
{
struct cpumask *user_mask = NULL;
swap(user_mask, p->user_cpus_ptr);
return user_mask;
}
void release_user_cpus_ptr(struct task_struct *p)
{
kfree(clear_user_cpus_ptr(p));
}
Then just use clear_user_cpus_ptr() in sched/core.c where we know what
we're doing (well, at least one of us does!).
From: Peter Zijlstra <peterz@infradead.org> Date: 2021-08-18 11:56:02
On Wed, Aug 18, 2021 at 12:56:41PM +0200, Peter Zijlstra wrote:
On Wed, Aug 18, 2021 at 11:42:28AM +0100, Will Deacon wrote:
quoted
I think the idea looks good, but perhaps we could wrap things up a bit:
/* Comment about why this is useful with RT */
static cpumask_t *clear_user_cpus_ptr(struct task_struct *p)
{
struct cpumask *user_mask = NULL;
swap(user_mask, p->user_cpus_ptr);
return user_mask;
}
void release_user_cpus_ptr(struct task_struct *p)
{
kfree(clear_user_cpus_ptr(p));
}
Then just use clear_user_cpus_ptr() in sched/core.c where we know what
we're doing (well, at least one of us does!).
From: Will Deacon <will@kernel.org> Date: 2021-08-18 12:19:14
On Wed, Aug 18, 2021 at 01:53:28PM +0200, Peter Zijlstra wrote:
On Wed, Aug 18, 2021 at 12:56:41PM +0200, Peter Zijlstra wrote:
quoted
On Wed, Aug 18, 2021 at 11:42:28AM +0100, Will Deacon wrote:
quoted
quoted
I think the idea looks good, but perhaps we could wrap things up a bit:
/* Comment about why this is useful with RT */
static cpumask_t *clear_user_cpus_ptr(struct task_struct *p)
{
struct cpumask *user_mask = NULL;
swap(user_mask, p->user_cpus_ptr);
return user_mask;
}
void release_user_cpus_ptr(struct task_struct *p)
{
kfree(clear_user_cpus_ptr(p));
}
Then just use clear_user_cpus_ptr() in sched/core.c where we know what
we're doing (well, at least one of us does!).
From: Peter Zijlstra <peterz@infradead.org> Date: 2021-08-18 11:08:38
On Wed, Aug 18, 2021 at 11:42:28AM +0100, Will Deacon wrote:
Ah, sorry. I didn't realise you couldn't _free_ with a raw lock held in RT.
Is there somewhere I can read up on that?
It's because the allocators use spinlock_t, which cannot nest inside
raw_spinlock_t.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
+ unsigned long flags;
+ struct cpumask *mask = p->user_cpus_ptr;
I've fixed all that up to be proper reverse x-mas trees; similar for
other patches.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Will Deacon <will@kernel.org> Date: 2021-07-30 11:25:34
In preparation for restricting the affinity of a task during execve()
on arm64, introduce a new dl_task_check_affinity() helper function to
give an indication as to whether the restricted mask is admissible for
a deadline task.
Reviewed-by: Daniel Bristot de Oliveira <redacted>
Signed-off-by: Will Deacon <will@kernel.org>
---
include/linux/sched.h | 6 ++++++
kernel/sched/core.c | 46 +++++++++++++++++++++++++++----------------
2 files changed, 35 insertions(+), 17 deletions(-)
From: Will Deacon <will@kernel.org> Date: 2021-07-30 11:25:40
Provide an implementation of task_cpu_possible_mask() so that we can
prevent 64-bit-only cores being added to the 'cpus_mask' for compat
tasks on systems with mismatched 32-bit support at EL0,
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
---
arch/arm64/include/asm/mmu_context.h | 13 +++++++++++++
1 file changed, 13 insertions(+)
From: Will Deacon <will@kernel.org> Date: 2021-07-30 11:25:47
When exec'ing a 32-bit task on a system with mismatched support for
32-bit EL0, try to ensure that it starts life on a CPU that can actually
run it.
Similarly, when exec'ing a 64-bit task on such a system, try to restore
the old affinity mask if it was previously restricted.
Reviewed-by: Daniel Bristot de Oliveira <redacted>
Reviewed-by: Quentin Perret <redacted>
Signed-off-by: Will Deacon <will@kernel.org>
---
arch/arm64/include/asm/elf.h | 6 ++----
arch/arm64/kernel/process.c | 39 +++++++++++++++++++++++++++++++++++-
2 files changed, 40 insertions(+), 5 deletions(-)
@@ -579,6 +580,28 @@ unsigned long arch_align_stack(unsigned long sp)returnsp&~0xf;}+#ifdef CONFIG_COMPAT+intcompat_elf_check_arch(conststructelf32_hdr*hdr)+{+if(!system_supports_32bit_el0())+returnfalse;++if((hdr)->e_machine!=EM_ARM)+returnfalse;++if(!((hdr)->e_flags&EF_ARM_EABI_MASK))+returnfalse;++/*+*Preventexecve()ofa32-bitprogramfromadeadlinetask+*iftherestrictedaffinitymaskwouldbeinadmissibleonan+*asymmetricsystem.+*/+return!static_branch_unlikely(&arm64_mismatched_32bit_el0)||+!dl_task_check_affinity(current,system_32bit_el0_cpumask());+}+#endif+/**Calledfromsetup_new_exec()after(COMPAT_)SET_PERSONALITY.*/
From: Will Deacon <will@kernel.org> Date: 2021-07-30 11:25:50
If we want to support 32-bit applications, then when we identify a CPU
with mismatched 32-bit EL0 support we must ensure that we will always
have an active 32-bit CPU available to us from then on. This is important
for the scheduler, because is_cpu_allowed() will be constrained to 32-bit
CPUs for compat tasks and forced migration due to a hotplug event will
hang if no 32-bit CPUs are available.
On detecting a mismatch, prevent offlining of either the mismatching CPU
if it is 32-bit capable, or find the first active 32-bit capable CPU
otherwise.
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
---
arch/arm64/kernel/cpufeature.c | 25 ++++++++++++++++++++++++-
1 file changed, 24 insertions(+), 1 deletion(-)
@@ -2900,15 +2900,38 @@ void __init setup_cpu_features(void)staticintenable_mismatched_32bit_el0(unsignedintcpu){+/*+*Thefirst32-bit-capableCPUwedetectedandsocannolonger+*beofflinedbyuserspace.-1indicateswehaven'tyetonlined+*a32-bit-capableCPU.+*/+staticintlucky_winner=-1;+structcpuinfo_arm64*info=&per_cpu(cpu_data,cpu);boolcpu_32bit=id_aa64pfr0_32bit_el0(info->reg_id_aa64pfr0);if(cpu_32bit){cpumask_set_cpu(cpu,cpu_32bit_el0_mask);static_branch_enable_cpuslocked(&arm64_mismatched_32bit_el0);-setup_elf_hwcaps(compat_elf_hwcaps);}+if(cpumask_test_cpu(0,cpu_32bit_el0_mask)==cpu_32bit)+return0;++if(lucky_winner>=0)+return0;++/*+*We'vedetectedamismatch.WeneedtokeeponeofourCPUswith+*32-bitEL0onlinesothatis_cpu_allowed()doesn'tenduprejecting+*everyCPUinthesystemfora32-bittask.+*/+lucky_winner=cpu_32bit?cpu:cpumask_any_and(cpu_32bit_el0_mask,+cpu_active_mask);+get_cpu_device(lucky_winner)->offline_disabled=true;+setup_elf_hwcaps(compat_elf_hwcaps);+pr_info("Asymmetric 32-bit EL0 support detected on CPU %u; CPU hot-unplug disabled on CPU %u\n",+cpu,lucky_winner);return0;}
--
2.32.0.402.g57bb445576-goog
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Will Deacon <will@kernel.org> Date: 2021-07-30 11:25:54
Since 32-bit applications will be killed if they are caught trying to
execute on a 64-bit-only CPU in a mismatched system, advertise the set
of 32-bit capable CPUs to userspace in sysfs.
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
---
.../ABI/testing/sysfs-devices-system-cpu | 9 +++++++++
arch/arm64/kernel/cpufeature.c | 19 +++++++++++++++++++
2 files changed, 28 insertions(+)
@@ -494,6 +494,15 @@ Description: AArch64 CPU registers 'identification' directory exposes the CPU ID registers for identifying model and revision of the CPU.+What: /sys/devices/system/cpu/aarch32_el0+Date: May 2021+Contact: Linux ARM Kernel Mailing list <linux-arm-kernel@lists.infradead.org>+Description: Identifies the subset of CPUs in the system that can execute+ AArch32 (32-bit ARM) applications. If present, the same format as+ /sys/devices/system/cpu/{offline,online,possible,present} is used.+ If absent, then all or none of the CPUs can execute AArch32+ applications and execve() will behave accordingly.+ What: /sys/devices/system/cpu/cpu#/cpu_capacity Date: December 2016 Contact: Linux kernel mailing list <linux-kernel@vger.kernel.org>
From: Will Deacon <will@kernel.org> Date: 2021-07-30 11:26:05
Allow systems with mismatched 32-bit support at EL0 to run 32-bit
applications based on a new kernel parameter.
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
---
Documentation/admin-guide/kernel-parameters.txt | 8 ++++++++
arch/arm64/kernel/cpufeature.c | 7 +++++++
2 files changed, 15 insertions(+)
@@ -287,6 +287,14 @@ do not want to use tracing_snapshot_alloc() as it needs to be done where GFP_KERNEL allocations are allowed.+ allow_mismatched_32bit_el0 [ARM64]+ Allow execve() of 32-bit applications and setting of the+ PER_LINUX32 personality on systems where only a strict+ subset of the CPUs support 32-bit EL0. When this+ parameter is present, the set of CPUs supporting 32-bit+ EL0 is indicated by /sys/devices/system/cpu/aarch32_el0+ and hot-unplug operations may be restricted.+ amd_iommu= [HW,X86-64] Pass parameters to the AMD IOMMU driver in the system. Possible values are:
From: Will Deacon <will@kernel.org> Date: 2021-07-30 11:26:08
The scheduler now knows enough about these braindead systems to place
32-bit tasks accordingly, so throw out the safety checks and allow the
ret-to-user path to avoid do_notify_resume() if there is nothing to do.
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
---
arch/arm64/kernel/process.c | 14 +-------------
arch/arm64/kernel/signal.c | 26 --------------------------
2 files changed, 1 insertion(+), 39 deletions(-)
From: Will Deacon <will@kernel.org> Date: 2021-07-30 11:26:11
Document support for running 32-bit tasks on asymmetric 32-bit systems
and its impact on the user ABI when enabled.
Signed-off-by: Will Deacon <will@kernel.org>
---
.../admin-guide/kernel-parameters.txt | 3 +
Documentation/arm64/asymmetric-32bit.rst | 155 ++++++++++++++++++
Documentation/arm64/index.rst | 1 +
3 files changed, 159 insertions(+)
create mode 100644 Documentation/arm64/asymmetric-32bit.rst
@@ -295,6 +295,9 @@ EL0 is indicated by /sys/devices/system/cpu/aarch32_el0 and hot-unplug operations may be restricted.+ See Documentation/arm64/asymmetric-32bit.rst for more+ information.+ amd_iommu= [HW,X86-64] Pass parameters to the AMD IOMMU driver in the system. Possible values are:
@@ -0,0 +1,155 @@+======================+Asymmetric 32-bit SoCs+======================++Author: Will Deacon <will@kernel.org>++This document describes the impact of asymmetric 32-bit SoCs on the+execution of 32-bit (``AArch32``) applications.++Date: 2021-05-17++Introduction+============++Some Armv9 SoCs suffer from a big.LITTLE misfeature where only a subset+of the CPUs are capable of executing 32-bit user applications. On such+a system, Linux by default treats the asymmetry as a "mismatch" and+disables support for both the ``PER_LINUX32`` personality and+``execve(2)`` of 32-bit ELF binaries, with the latter returning+``-ENOEXEC``. If the mismatch is detected during late onlining of a+64-bit-only CPU, then the onlining operation fails and the new CPU is+unavailable for scheduling.++Surprisingly, these SoCs have been produced with the intention of+running legacy 32-bit binaries. Unsurprisingly, that doesn't work very+well with the default behaviour of Linux.++It seems inevitable that future SoCs will drop 32-bit support+altogether, so if you're stuck in the unenviable position of needing to+run 32-bit code on one of these transitionary platforms then you would+be wise to consider alternatives such as recompilation, emulation or+retirement. If neither of those options are practical, then read on.++Enabling kernel support+=======================++Since the kernel support is not completely transparent to userspace,+allowing 32-bit tasks to run on an asymmetric 32-bit system requires an+explicit "opt-in" and can be enabled by passing the+``allow_mismatched_32bit_el0`` parameter on the kernel command-line.++For the remainder of this document we will refer to an *asymmetric+system* to mean an asymmetric 32-bit SoC running Linux with this kernel+command-line option enabled.++Userspace impact+================++32-bit tasks running on an asymmetric system behave in mostly the same+way as on a homogeneous system, with a few key differences relating to+CPU affinity.++sysfs+-----++The subset of CPUs capable of running 32-bit tasks is described in+``/sys/devices/system/cpu/aarch32_el0`` and is documented further in+``Documentation/ABI/testing/sysfs-devices-system-cpu``.++**Note:** CPUs are advertised by this file as they are detected and so+late-onlining of 32-bit-capable CPUs can result in the file contents+being modified by the kernel at runtime. Once advertised, CPUs are never+removed from the file.++``execve(2)``+-------------++On a homogeneous system, the CPU affinity of a task is preserved across+``execve(2)``. This is not always possible on an asymmetric system,+specifically when the new program being executed is 32-bit yet the+affinity mask contains 64-bit-only CPUs. In this situation, the kernel+determines the new affinity mask as follows:++1. If the 32-bit-capable subset of the affinity mask is not empty,+ then the affinity is restricted to that subset and the old affinity+ mask is saved. This saved mask is inherited over ``fork(2)`` and+ preserved across ``execve(2)`` of 32-bit programs.++**Note:** This step does not apply to ``SCHED_DEADLINE`` tasks.+ See `SCHED_DEADLINE`_.++2. Otherwise, the cpuset hierarchy of the task is walked until an+ ancestor is found containing at least one 32-bit-capable CPU. The+ affinity of the task is then changed to match the 32-bit-capable+ subset of the cpuset determined by the walk.++3. On failure (i.e. out of memory), the affinity is changed to the set+ of all 32-bit-capable CPUs of which the kernel is aware.++A subsequent ``execve(2)`` of a 64-bit program by the 32-bit task will+invalidate the affinity mask saved in (1) and attempt to restore the CPU+affinity of the task using the saved mask if it was previously valid.+This restoration may fail due to intervening changes to the deadline+policy or cpuset hierarchy, in which case the ``execve(2)`` continues+with the affinity unchanged.++Calls to ``sched_setaffinity(2)`` for a 32-bit task will consider only+the 32-bit-capable CPUs of the requested affinity mask. On success, the+affinity for the task is updated and any saved mask from a prior+``execve(2)`` is invalidated.++``SCHED_DEADLINE``+------------------++Explicit admission of a 32-bit deadline task to the default root domain+(e.g. by calling ``sched_setattr(2)``) is rejected on an asymmetric+32-bit system unless admission control is disabled by writing -1 to+``/proc/sys/kernel/sched_rt_runtime_us``.++``execve(2)`` of a 32-bit program from a 64-bit deadline task will+return ``-ENOEXEC`` if the root domain for the task contains any+64-bit-only CPUs and admission control is enabled. Concurrent offlining+of 32-bit-capable CPUs may still necessitate the procedure described in+`execve(2)`_, in which case step (1) is skipped and a warning is+emitted on the console.++**Note:** It is recommended that a set of 32-bit-capable CPUs are placed+into a separate root domain if ``SCHED_DEADLINE`` is to be used with+32-bit tasks on an asymmetric system. Failure to do so is likely to+result in missed deadlines.++Cpusets+-------++The affinity of a 32-bit task on an asymmetric system may include CPUs+that are not explicitly allowed by the cpuset to which it is attached.+This can occur as a result of the following two situations:++- A 64-bit task attached to a cpuset which allows only 64-bit CPUs+ executes a 32-bit program.++- All of the 32-bit-capable CPUs allowed by a cpuset containing a+ 32-bit task are offlined.++In both of these cases, the new affinity is calculated according to step+(2) of the process described in `execve(2)`_ and the cpuset hierarchy is+unchanged irrespective of the cgroup version.++CPU hotplug+-----------++On an asymmetric system, the first detected 32-bit-capable CPU is+prevented from being offlined by userspace and any such attempt will+return ``-EPERM``. Note that suspend is still permitted even if the+primary CPU (i.e. CPU 0) is 64-bit-only.++KVM+---++Although KVM will not advertise 32-bit EL0 support to any vCPUs on an+asymmetric system, a broken guest at EL1 could still attempt to execute+32-bit code at EL0. In this case, an exit from a vCPU thread in 32-bit+mode will return to host userspace with an ``exit_reason`` of+``KVM_EXIT_FAIL_ENTRY`` and will remain non-runnable until successfully+re-initialised by a subsequent ``KVM_ARM_VCPU_INIT`` operation.