Two major parts to this series:
1. Removal of the extended cede offline mode for CPUs as well as the
partition suspend code which accommodates it by temporarily
onlining all CPUs prior to suspending the LPAR. This solves some
accounting problems, simplifies the pseries CPU hotplug code, and
greatly uncomplicates the existing partition suspend code, easing
a much-needed transition to the Linux suspend framework. The two
patches which make up this part have been posted before:
https://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=180718
and they are simply incorporated unchanged into the larger series
here, with Gautham's Reviewed-by added to patch #1.
2. Removal of the long-disabled "topology update" code, most of which
resides in mm/numa.c, but there are pieces in pseries and rtasd to
excise as well. This code was an attempt to honor changes in a
partition's NUMA properties arising from resource reassignments
which occur as part of a migration, VPHN change, or a Dynamic
Platform Optimizer operation. Its main technique is to remove and
re-add affected processors and LMBs and hope in vain that the
changes in cpu-node and physaddr-node relationships aren't
disruptive. We want to provide user space with some indication that
Linux's logical NUMA representation has become out of sync with the
platform's assignments, but we need to get this unusable stuff out
of the way before this code can sustain new features.
Nathan Lynch (18):
powerpc/pseries: remove cede offline state for CPUs
powerpc/rtas: don't online CPUs for partition suspend
powerpc/numa: remove ability to enable topology updates
powerpc/numa: remove unreachable topology update code
powerpc/numa: make vphn_enabled, prrn_enabled flags const
powerpc/numa: remove unreachable topology timer code
powerpc/numa: remove unreachable topology workqueue code
powerpc/numa: remove vphn_enabled and prrn_enabled internal flags
powerpc/numa: stub out numa_update_cpu_topology()
powerpc/numa: remove timed_topology_update()
powerpc/numa: remove start/stop_topology_update()
powerpc/rtasd: simplify handle_rtas_event(), emit message on events
powerpc/numa: remove prrn_is_enabled()
powerpc/numa: remove arch_update_cpu_topology
powerpc/pseries: remove prrn special case from DT update path
powerpc/pseries: remove memory "re-add" implementation
powerpc/pseries: remove dlpar_cpu_readd()
powerpc/pseries: remove obsolete memory hotplug DT notifier code
Documentation/core-api/cpu_hotplug.rst | 7 -
arch/powerpc/include/asm/rtas.h | 3 -
arch/powerpc/include/asm/topology.h | 27 -
arch/powerpc/kernel/rtas.c | 122 +----
arch/powerpc/kernel/rtasd.c | 28 +-
arch/powerpc/mm/numa.c | 486 ------------------
arch/powerpc/platforms/pseries/hotplug-cpu.c | 189 +------
.../platforms/pseries/hotplug-memory.c | 107 +---
arch/powerpc/platforms/pseries/mobility.c | 31 --
.../platforms/pseries/offline_states.h | 38 --
arch/powerpc/platforms/pseries/pmem.c | 1 -
arch/powerpc/platforms/pseries/smp.c | 28 +-
arch/powerpc/platforms/pseries/suspend.c | 27 +-
13 files changed, 22 insertions(+), 1072 deletions(-)
delete mode 100644 arch/powerpc/platforms/pseries/offline_states.h
--
2.25.4
This effectively reverts commit 3aa565f53c39 ("powerpc/pseries: Add
hooks to put the CPU into an appropriate offline state"), which added
an offline mode for CPUs which uses the H_CEDE hcall instead of the
architected stop-self RTAS function in order to facilitate "folding"
of dedicated mode processors on PowerVM platforms to achieve energy
savings. This has been the default offline mode since its
introduction.
There's nothing about stop-self that would prevent the hypervisor from
achieving the energy savings available via H_CEDE, so the original
premise of this change appears to be flawed.
I also have encountered the claim that the transition to and from
ceded state is much faster than stop-self/start-cpu. Certainly we
would not want to use stop-self as an *idle* mode. That is what H_CEDE
is for. However, this difference is insignificant in the context of
Linux CPU hotplug, where the latency of an offline or online operation
on current systems is on the order of 100ms, mainly attributable to
all the various subsystems' cpuhp callbacks.
The cede offline mode also prevents accurate accounting, as discussed
before:
https://lore.kernel.org/linuxppc-dev/1571740391-3251-1-git-send-email-ego@linux.vnet.ibm.com/
Unconditionally use stop-self to offline processor threads. This is
the architected method for offlining CPUs on PAPR systems.
The "cede_offline" boot parameter is rendered obsolete.
Removing this code enables the removal of the partition suspend code
which temporarily onlines all present CPUs.
Fixes: 3aa565f53c39 ("powerpc/pseries: Add hooks to put the CPU into an appropriate offline state")
Signed-off-by: Nathan Lynch <redacted>
Reviewed-by: Gautham R. Shenoy <redacted>
---
Documentation/core-api/cpu_hotplug.rst | 7 -
arch/powerpc/platforms/pseries/hotplug-cpu.c | 170 ++----------------
.../platforms/pseries/offline_states.h | 38 ----
arch/powerpc/platforms/pseries/pmem.c | 1 -
arch/powerpc/platforms/pseries/smp.c | 28 +--
5 files changed, 15 insertions(+), 229 deletions(-)
delete mode 100644 arch/powerpc/platforms/pseries/offline_states.h
@@ -50,13 +50,6 @@ Command Line Switches This option is limited to the X86 and S390 architecture.-``cede_offline={"off","on"}``- Use this option to disable/enable putting offlined processors to an extended-``H_CEDE`` state on supported pseries platforms. If nothing is specified,-``cede_offline`` is set to "on".-- This option is limited to the PowerPC architecture.-``cpu0_hotplug`` Allow to shutdown CPU0.
@@ -35,54 +35,10 @@#include<asm/topology.h>#include"pseries.h"-#include"offline_states.h"/* This version can't take the spinlock, because it never returns */staticintrtas_stop_self_token=RTAS_UNKNOWN_SERVICE;-staticDEFINE_PER_CPU(enumcpu_state_vals,preferred_offline_state)=-CPU_STATE_OFFLINE;-staticDEFINE_PER_CPU(enumcpu_state_vals,current_state)=CPU_STATE_OFFLINE;--staticenumcpu_state_valsdefault_offline_state=CPU_STATE_OFFLINE;--staticboolcede_offline_enabled__read_mostly=true;--/*-*Enable/disablecede_offlinewhenavailable.-*/-staticint__initsetup_cede_offline(char*str)-{-return(kstrtobool(str,&cede_offline_enabled)==0);-}--__setup("cede_offline=",setup_cede_offline);--enumcpu_state_valsget_cpu_current_state(intcpu)-{-returnper_cpu(current_state,cpu);-}--voidset_cpu_current_state(intcpu,enumcpu_state_valsstate)-{-per_cpu(current_state,cpu)=state;-}--enumcpu_state_valsget_preferred_offline_state(intcpu)-{-returnper_cpu(preferred_offline_state,cpu);-}--voidset_preferred_offline_state(intcpu,enumcpu_state_valsstate)-{-per_cpu(preferred_offline_state,cpu)=state;-}--voidset_default_offline_state(intcpu)-{-per_cpu(preferred_offline_state,cpu)=default_offline_state;-}-staticvoidrtas_stop_self(void){staticstructrtas_argsargs;
@@ -112,49 +66,6 @@ static void pseries_mach_cpu_die(void)elsexics_teardown_cpu();-if(get_preferred_offline_state(cpu)==CPU_STATE_INACTIVE){-set_cpu_current_state(cpu,CPU_STATE_INACTIVE);-if(ppc_md.suspend_disable_cpu)-ppc_md.suspend_disable_cpu();--cede_latency_hint=2;--get_lppaca()->idle=1;-if(!lppaca_shared_proc(get_lppaca()))-get_lppaca()->donate_dedicated_cpu=1;--while(get_preferred_offline_state(cpu)==CPU_STATE_INACTIVE){-while(!prep_irq_for_idle()){-local_irq_enable();-local_irq_disable();-}--extended_cede_processor(cede_latency_hint);-}--local_irq_disable();--if(!lppaca_shared_proc(get_lppaca()))-get_lppaca()->donate_dedicated_cpu=0;-get_lppaca()->idle=0;--if(get_preferred_offline_state(cpu)==CPU_STATE_ONLINE){-unregister_slb_shadow(hwcpu);--hard_irq_disable();-/*-*Calltostart_secondary_resume()willnotreturn.-*Kernelstackwillberesetandstart_secondary()-*willbecalledtocontinuetheonlineoperation.-*/-start_secondary_resume();-}-}--/* Requested state is CPU_STATE_OFFLINE at this point */-WARN_ON(get_preferred_offline_state(cpu)!=CPU_STATE_OFFLINE);--set_cpu_current_state(cpu,CPU_STATE_OFFLINE);unregister_slb_shadow(hwcpu);rtas_stop_self();
@@ -200,24 +111,13 @@ static void pseries_cpu_die(unsigned int cpu)intcpu_status=1;unsignedintpcpu=get_hard_smp_processor_id(cpu);-if(get_preferred_offline_state(cpu)==CPU_STATE_INACTIVE){-cpu_status=1;-for(tries=0;tries<5000;tries++){-if(get_cpu_current_state(cpu)==CPU_STATE_INACTIVE){-cpu_status=0;-break;-}-msleep(1);-}-}elseif(get_preferred_offline_state(cpu)==CPU_STATE_OFFLINE){+for(tries=0;tries<25;tries++){+cpu_status=smp_query_cpu_stopped(pcpu);+if(cpu_status==QCSS_STOPPED||+cpu_status==QCSS_HARDWARE_ERROR)+break;+cpu_relax();-for(tries=0;tries<25;tries++){-cpu_status=smp_query_cpu_stopped(pcpu);-if(cpu_status==QCSS_STOPPED||-cpu_status==QCSS_HARDWARE_ERROR)-break;-cpu_relax();-}}if(cpu_status!=0){
@@ -359,28 +259,15 @@ static int dlpar_offline_cpu(struct device_node *dn)if(get_hard_smp_processor_id(cpu)!=thread)continue;-if(get_cpu_current_state(cpu)==CPU_STATE_OFFLINE)+if(!cpu_online(cpu))break;-if(get_cpu_current_state(cpu)==CPU_STATE_ONLINE){-set_preferred_offline_state(cpu,-CPU_STATE_OFFLINE);-cpu_maps_update_done();-timed_topology_update(1);-rc=device_offline(get_cpu_device(cpu));-if(rc)-gotoout;-cpu_maps_update_begin();-break;-}--/*-*ThecpuisinCPU_STATE_INACTIVE.-*Upgradeit'sstatetoCPU_STATE_OFFLINE.-*/-set_preferred_offline_state(cpu,CPU_STATE_OFFLINE);-WARN_ON(plpar_hcall_norets(H_PROD,thread)!=H_SUCCESS);-__cpu_die(cpu);+cpu_maps_update_done();+timed_topology_update(1);+rc=device_offline(get_cpu_device(cpu));+if(rc)+gotoout;+cpu_maps_update_begin();break;}if(cpu==num_possible_cpus()){
@@ -414,8 +301,6 @@ static int dlpar_online_cpu(struct device_node *dn)for_each_present_cpu(cpu){if(get_hard_smp_processor_id(cpu)!=thread)continue;-BUG_ON(get_cpu_current_state(cpu)-!=CPU_STATE_OFFLINE);cpu_maps_update_done();timed_topology_update(1);find_and_online_cpu_nid(cpu);
@@ -1056,16 +922,8 @@ static int __init pseries_cpu_hotplug_init(void)smp_ops->cpu_die=pseries_cpu_die;/* Processors can be added/removed only on LPAR */-if(firmware_has_feature(FW_FEATURE_LPAR)){+if(firmware_has_feature(FW_FEATURE_LPAR))of_reconfig_notifier_register(&pseries_smp_nb);-cpu_maps_update_begin();-if(cede_offline_enabled&&parse_cede_parameters()==0){-default_offline_state=CPU_STATE_INACTIVE;-for_each_online_cpu(cpu)-set_default_offline_state(cpu);-}-cpu_maps_update_done();-}return0;}
@@ -163,20 +151,6 @@ static int smp_pSeries_kick_cpu(int nr)*theprocessorwillcontinueontosecondary_start*/paca_ptrs[nr]->cpu_start=1;-#ifdef CONFIG_HOTPLUG_CPU-set_preferred_offline_state(nr,CPU_STATE_ONLINE);--if(get_cpu_current_state(nr)==CPU_STATE_INACTIVE){-longrc;-unsignedlonghcpuid;--hcpuid=get_hard_smp_processor_id(nr);-rc=plpar_hcall_norets(H_PROD,hcpuid);-if(rc!=H_SUCCESS)-printk(KERN_ERR"Error: Prod to wake up processor %d "-"Ret= %ld\n",nr,rc);-}-#endifreturn0;}
Remove the /proc/powerpc/topology_updates interface and the
topology_updates=on/off command line argument. The internal
topology_updates_enabled flag remains for now, but always false.
Signed-off-by: Nathan Lynch <redacted>
---
arch/powerpc/mm/numa.c | 71 +-----------------------------------------
1 file changed, 1 insertion(+), 70 deletions(-)
@@ -1632,52 +1612,6 @@ int prrn_is_enabled(void)returnprrn_enabled;}-staticinttopology_read(structseq_file*file,void*v)-{-if(vphn_enabled||prrn_enabled)-seq_puts(file,"on\n");-else-seq_puts(file,"off\n");--return0;-}--staticinttopology_open(structinode*inode,structfile*file)-{-returnsingle_open(file,topology_read,NULL);-}--staticssize_ttopology_write(structfile*file,constchar__user*buf,-size_tcount,loff_t*off)-{-charkbuf[4];/* "on" or "off" plus null. */-intread_len;--read_len=count<3?count:3;-if(copy_from_user(kbuf,buf,read_len))-return-EINVAL;--kbuf[read_len]='\0';--if(!strncmp(kbuf,"on",2)){-topology_updates_enabled=true;-start_topology_update();-}elseif(!strncmp(kbuf,"off",3)){-stop_topology_update();-topology_updates_enabled=false;-}else-return-EINVAL;--returncount;-}--staticconststructproc_opstopology_proc_ops={-.proc_read=seq_read,-.proc_write=topology_write,-.proc_open=topology_open,-.proc_release=single_release,-};-staticinttopology_update_init(void){start_topology_update();
@@ -1685,9 +1619,6 @@ static int topology_update_init(void)if(vphn_enabled)topology_schedule_update();-if(!proc_create("powerpc/topology_updates",0644,NULL,&topology_proc_ops))-return-ENOMEM;-topology_inited=1;return0;}
Partition suspension, used for hibernation and migration, requires
that the OS place all but one of the LPAR's processor threads into one
of two states prior to calling the ibm,suspend-me RTAS function:
* the architected offline state (via RTAS stop-self); or
* the H_JOIN hcall, which does not return until the partition
resumes execution
Using H_CEDE as the offline mode, introduced by
commit 3aa565f53c39 ("powerpc/pseries: Add hooks to put the CPU into
an appropriate offline state"), means that any threads which are
offline from Linux's point of view must be moved to one of those two
states before a partition suspension can proceed.
This was eventually addressed in commit 120496ac2d2d ("powerpc: Bring
all threads online prior to migration/hibernation"), which added code
to temporarily bring up any offline processor threads so they can call
H_JOIN. Conceptually this is fine, but the implementation has had
multiple races with cpu hotplug operations initiated from user
space[1][2][3], the error handling is fragile, and it generates
user-visible cpu hotplug events which is a lot of noise for a platform
feature that's supposed to minimize disruption to workloads.
With commit 3aa565f53c39 ("powerpc/pseries: Add hooks to put the CPU
into an appropriate offline state") reverted, this code becomes
unnecessary, so remove it. Since any offline CPUs now are truly
offline from the platform's point of view, it is no longer necessary
to bring up CPUs only to have them call H_JOIN and then go offline
again upon resuming. Only active threads are required to call H_JOIN;
stopped threads can be left alone.
[1] commit a6717c01ddc2 ("powerpc/rtas: use device model APIs and
serialization during LPM")
[2] commit 9fb603050ffd ("powerpc/rtas: retry when cpu offline races
with suspend/migration")
[3] commit dfd718a2ed1f ("powerpc/rtas: Fix a potential race between
CPU-Offline & Migration")
Fixes: 120496ac2d2d ("powerpc: Bring all threads online prior to migration/hibernation")
Signed-off-by: Nathan Lynch <redacted>
---
arch/powerpc/include/asm/rtas.h | 2 -
arch/powerpc/kernel/rtas.c | 122 +----------------------
arch/powerpc/platforms/pseries/suspend.c | 22 +---
3 files changed, 3 insertions(+), 143 deletions(-)
@@ -253,8 +253,6 @@ extern int rtas_set_indicator_fast(int indicator, int index, int new_value);externvoidrtas_progress(char*s,unsignedshorthex);externintrtas_suspend_cpu(structrtas_suspend_me_data*data);externintrtas_suspend_last_cpu(structrtas_suspend_me_data*data);-externintrtas_online_cpus_mask(cpumask_var_tcpus);-externintrtas_offline_cpus_mask(cpumask_var_tcpus);externintrtas_ibm_suspend_me(u64handle);structrtc_time;
@@ -843,96 +843,6 @@ static void rtas_percpu_suspend_me(void *info)__rtas_suspend_cpu((structrtas_suspend_me_data*)info,1);}-enumrtas_cpu_state{-DOWN,-UP,-};--#ifndef CONFIG_SMP-staticintrtas_cpu_state_change_mask(enumrtas_cpu_statestate,-cpumask_var_tcpus)-{-if(!cpumask_empty(cpus)){-cpumask_clear(cpus);-return-EINVAL;-}else-return0;-}-#else-/* On return cpumask will be altered to indicate CPUs changed.-*CPUswithstateschangedwillbesetinthemask,-*CPUswithstatusunchangedwillbeunsetinthemask.*/-staticintrtas_cpu_state_change_mask(enumrtas_cpu_statestate,-cpumask_var_tcpus)-{-intcpu;-intcpuret=0;-intret=0;--if(cpumask_empty(cpus))-return0;--for_each_cpu(cpu,cpus){-structdevice*dev=get_cpu_device(cpu);--switch(state){-caseDOWN:-cpuret=device_offline(dev);-break;-caseUP:-cpuret=device_online(dev);-break;-}-if(cpuret<0){-pr_debug("%s: cpu_%s for cpu#%d returned %d.\n",-__func__,-((state==UP)?"up":"down"),-cpu,cpuret);-if(!ret)-ret=cpuret;-if(state==UP){-/* clear bits for unchanged cpus, return */-cpumask_shift_right(cpus,cpus,cpu);-cpumask_shift_left(cpus,cpus,cpu);-break;-}else{-/* clear bit for unchanged cpu, continue */-cpumask_clear_cpu(cpu,cpus);-}-}-cond_resched();-}--returnret;-}-#endif--intrtas_online_cpus_mask(cpumask_var_tcpus)-{-intret;--ret=rtas_cpu_state_change_mask(UP,cpus);--if(ret){-cpumask_var_ttmp_mask;--if(!alloc_cpumask_var(&tmp_mask,GFP_KERNEL))-returnret;--/* Use tmp_mask to preserve cpus mask from first failure */-cpumask_copy(tmp_mask,cpus);-rtas_offline_cpus_mask(tmp_mask);-free_cpumask_var(tmp_mask);-}--returnret;-}--intrtas_offline_cpus_mask(cpumask_var_tcpus)-{-returnrtas_cpu_state_change_mask(DOWN,cpus);-}-intrtas_ibm_suspend_me(u64handle){longstate;
@@ -940,8 +850,6 @@ int rtas_ibm_suspend_me(u64 handle)unsignedlongretbuf[PLPAR_HCALL_BUFSIZE];structrtas_suspend_me_datadata;DECLARE_COMPLETION_ONSTACK(done);-cpumask_var_toffline_mask;-intcpuret;if(!rtas_service_present("ibm,suspend-me"))return-ENOSYS;
@@ -962,9 +870,6 @@ int rtas_ibm_suspend_me(u64 handle)return-EIO;}-if(!alloc_cpumask_var(&offline_mask,GFP_KERNEL))-return-ENOMEM;-atomic_set(&data.working,0);atomic_set(&data.done,0);atomic_set(&data.error,0);
@@ -973,24 +878,8 @@ int rtas_ibm_suspend_me(u64 handle)lock_device_hotplug();-/* All present CPUs must be online */-cpumask_andnot(offline_mask,cpu_present_mask,cpu_online_mask);-cpuret=rtas_online_cpus_mask(offline_mask);-if(cpuret){-pr_err("%s: Could not bring present CPUs online.\n",__func__);-atomic_set(&data.error,cpuret);-gotoout;-}-cpu_hotplug_disable();-/* Check if we raced with a CPU-Offline Operation */-if(!cpumask_equal(cpu_present_mask,cpu_online_mask)){-pr_info("%s: Raced against a concurrent CPU-Offline\n",__func__);-atomic_set(&data.error,-EAGAIN);-gotoout_hotplug_enable;-}-/* Call function on all CPUs. One of us will make the*rtascall*/
@@ -1001,18 +890,11 @@ int rtas_ibm_suspend_me(u64 handle)if(atomic_read(&data.error)!=0)printk(KERN_ERR"Error doing global join\n");-out_hotplug_enable:-cpu_hotplug_enable();-/* Take down CPUs not online prior to suspend */-cpuret=rtas_offline_cpus_mask(offline_mask);-if(cpuret)-pr_warn("%s: Could not restore CPUs to offline state.\n",-__func__);+cpu_hotplug_enable();-out:unlock_device_hotplug();-free_cpumask_var(offline_mask);+returnatomic_read(&data.error);}
@@ -150,32 +146,16 @@ static ssize_t store_hibernate(struct device *dev,}while(rc==-EAGAIN);if(!rc){-/* All present CPUs must be online */-cpumask_andnot(offline_mask,cpu_present_mask,-cpu_online_mask);-rc=rtas_online_cpus_mask(offline_mask);-if(rc){-pr_err("%s: Could not bring present CPUs online.\n",-__func__);-gotoout;-}-stop_topology_update();rc=pm_suspend(PM_SUSPEND_MEM);start_topology_update();--/* Take down CPUs not online prior to suspend */-if(!rtas_offline_cpus_mask(offline_mask))-pr_warn("%s: Could not restore CPUs to offline "-"state.\n",__func__);}stream_id=0;if(!rc)rc=count;-out:-free_cpumask_var(offline_mask);+returnrc;}
Since the topology_updates_enabled flag is now always false, remove it
and the code which has become unreachable. This is the minimum change
that prevents 'defined but unused' warnings emitted by the compiler
after stubbing out the start/stop_topology_updates() functions.
Signed-off-by: Nathan Lynch <redacted>
---
arch/powerpc/mm/numa.c | 149 +----------------------------------------
1 file changed, 2 insertions(+), 147 deletions(-)
@@ -1158,63 +1155,6 @@ int timed_topology_update(int nsecs)return0;}-/*-*Storethecurrentvaluesoftheassociativitychangecountersinthe-*hypervisor.-*/-staticvoidsetup_cpu_associativity_change_counters(void)-{-intcpu;--/* The VPHN feature supports a maximum of 8 reference points */-BUILD_BUG_ON(MAX_DISTANCE_REF_POINTS>8);--for_each_possible_cpu(cpu){-inti;-u8*counts=vphn_cpu_change_counts[cpu];-volatileu8*hypervisor_counts=lppaca_of(cpu).vphn_assoc_counts;--for(i=0;i<distance_ref_points_depth;i++)-counts[i]=hypervisor_counts[i];-}-}--/*-*Thehypervisormaintainsasetof8associativitychangecountersin-*theVPAofeachcputhatcorrespondtotheassociativitylevelsinthe-*ibm,associativity-reference-pointsproperty.Whenanassociativity-*levelchanges,thecorrespondingcounterisincremented.-*-*Setabitincpu_associativity_changes_maskforeachcpuwhosehome-*nodeassociativitylevelshavechanged.-*-*Returnsthenumberofcpuswithunhandledassociativitychanges.-*/-staticintupdate_cpu_associativity_changes_mask(void)-{-intcpu;-cpumask_t*changes=&cpu_associativity_changes_mask;--for_each_possible_cpu(cpu){-inti,changed=0;-u8*counts=vphn_cpu_change_counts[cpu];-volatileu8*hypervisor_counts=lppaca_of(cpu).vphn_assoc_counts;--for(i=0;i<distance_ref_points_depth;i++){-if(hypervisor_counts[i]!=counts[i]){-counts[i]=hypervisor_counts[i];-changed=1;-}-}-if(changed){-cpumask_or(changes,changes,cpu_sibling_mask(cpu));-cpu=cpu_last_thread_sibling(cpu);-}-}--returncpumask_weight(changes);-}-/**Retrievethenewassociativityinformationforavirtualprocessor's*homenode.
Previous changes have made it so these flags are never changed;
enforce this by making them const.
Signed-off-by: Nathan Lynch <redacted>
---
arch/powerpc/mm/numa.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
Since vphn_enabled is always 0, we can stub out
timed_topology_update() and remove the code which becomes unreachable.
Signed-off-by: Nathan Lynch <redacted>
---
arch/powerpc/mm/numa.c | 21 ---------------------
1 file changed, 21 deletions(-)
Since vphn_enabled is always 0, we can remove the call to
topology_schedule_update() and remove the code which becomes
unreachable as a result.
Signed-off-by: Nathan Lynch <redacted>
---
arch/powerpc/mm/numa.c | 14 --------------
1 file changed, 14 deletions(-)
These flags are always zero now; remove them and suitably adjust the
remaining references to them.
Signed-off-by: Nathan Lynch <redacted>
---
arch/powerpc/mm/numa.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
Previous changes have removed the code which sets bits in
cpu_associativity_changes_mask and thus it is never modifed at
runtime. From this we can reason that numa_update_cpu_topology()
always returns 0 without doing anything. Remove the body of
numa_update_cpu_topology() and remove all code which becomes
unreachable as a result.
Signed-off-by: Nathan Lynch <redacted>
---
arch/powerpc/mm/numa.c | 193 +----------------------------------------
1 file changed, 1 insertion(+), 192 deletions(-)
@@ -1122,14 +1122,6 @@ u64 memory_hotplug_max(void)/* Virtual Processor Home Node (VPHN) support */#ifdef CONFIG_PPC_SPLPAR-structtopology_update_data{-structtopology_update_data*next;-unsignedintcpu;-intold_nid;-intnew_nid;-};--staticcpumask_tcpu_associativity_changes_mask;staticinttopology_inited;/*
@@ -1219,192 +1211,9 @@ int find_and_online_cpu_nid(int cpu)returnnew_nid;}-/*-*UpdatetheCPUmapsandsysfsentriesforasingleCPUwhenitsNUMA-*characteristicschange.Thisfunctiondoesn'tperformanylockingandis-*onlysafetocallfromstop_machine().-*/-staticintupdate_cpu_topology(void*data)-{-structtopology_update_data*update;-unsignedlongcpu;--if(!data)-return-EINVAL;--cpu=smp_processor_id();--for(update=data;update;update=update->next){-intnew_nid=update->new_nid;-if(cpu!=update->cpu)-continue;--unmap_cpu_from_node(cpu);-map_cpu_to_node(cpu,new_nid);-set_cpu_numa_node(cpu,new_nid);-set_cpu_numa_mem(cpu,local_memory_node(new_nid));-vdso_getcpu_init();-}--return0;-}--staticintupdate_lookup_table(void*data)-{-structtopology_update_data*update;--if(!data)-return-EINVAL;--/*-*Upontopologyupdate,thenuma-cpulookuptableneedstobeupdated-*forallthreadsinthecore,includingofflineCPUs,toensurethat-*futurehotplugoperationsrespectthecpu-to-nodeassociativity-*properly.-*/-for(update=data;update;update=update->next){-intnid,base,j;--nid=update->new_nid;-base=cpu_first_thread_sibling(update->cpu);--for(j=0;j<threads_per_core;j++){-update_numa_cpu_lookup_table(base+j,nid);-}-}--return0;-}--/*-*Updatethenodemapsandsysfsentriesforeachcpuwhosehomenode-*haschanged.Returns1whenthetopologyhaschanged,and0otherwise.-*-*cpus_lockedsayswhetherwealreadyholdcpu_hotplug_lock.-*/intnuma_update_cpu_topology(boolcpus_locked){-unsignedintcpu,sibling,changed=0;-structtopology_update_data*updates,*ud;-cpumask_tupdated_cpus;-structdevice*dev;-intweight,new_nid,i=0;--if(topology_inited)-return0;--weight=cpumask_weight(&cpu_associativity_changes_mask);-if(!weight)-return0;--updates=kcalloc(weight,sizeof(*updates),GFP_KERNEL);-if(!updates)-return0;--cpumask_clear(&updated_cpus);--for_each_cpu(cpu,&cpu_associativity_changes_mask){-/*-*Ifsiblingsaren'tflaggedforchanges,updateslist-*willbetooshort.Skiponthisupdateandsetfornext-*update.-*/-if(!cpumask_subset(cpu_sibling_mask(cpu),-&cpu_associativity_changes_mask)){-pr_info("Sibling bits not set for associativity "-"change, cpu%d\n",cpu);-cpumask_or(&cpu_associativity_changes_mask,-&cpu_associativity_changes_mask,-cpu_sibling_mask(cpu));-cpu=cpu_last_thread_sibling(cpu);-continue;-}--new_nid=find_and_online_cpu_nid(cpu);--if(new_nid==numa_cpu_lookup_table[cpu]){-cpumask_andnot(&cpu_associativity_changes_mask,-&cpu_associativity_changes_mask,-cpu_sibling_mask(cpu));-dbg("Assoc chg gives same node %d for cpu%d\n",-new_nid,cpu);-cpu=cpu_last_thread_sibling(cpu);-continue;-}--for_each_cpu(sibling,cpu_sibling_mask(cpu)){-ud=&updates[i++];-ud->next=&updates[i];-ud->cpu=sibling;-ud->new_nid=new_nid;-ud->old_nid=numa_cpu_lookup_table[sibling];-cpumask_set_cpu(sibling,&updated_cpus);-}-cpu=cpu_last_thread_sibling(cpu);-}--/*-*Preventprocessingof'updates'fromoverflowingarray-*wherelastentryfilledina'next'pointer.-*/-if(i)-updates[i-1].next=NULL;--pr_debug("Topology update for the following CPUs:\n");-if(cpumask_weight(&updated_cpus)){-for(ud=&updates[0];ud;ud=ud->next){-pr_debug("cpu %d moving from node %d "-"to %d\n",ud->cpu,-ud->old_nid,ud->new_nid);-}-}--/*-*Incaseswherewehavenothingtoupdate(becausetheupdateslist-*istooshortorbecausethenewtopologyissameastheoldone),-*skipinvokingupdate_cpu_topology()viastop-machine().Thisis-*necessary(andnotjustafast-pathoptimization)sincestop-machine-*canendupelectingarandomCPUtorunupdate_cpu_topology(),and-*thustrickusintosettingupincorrectcpu-nodemappings(since-*'updates'iskzalloc()'ed).-*-*Andforthesimilarreason,wewillskipallthefollowingupdating.-*/-if(!cpumask_weight(&updated_cpus))-gotoout;--if(cpus_locked)-stop_machine_cpuslocked(update_cpu_topology,&updates[0],-&updated_cpus);-else-stop_machine(update_cpu_topology,&updates[0],&updated_cpus);--/*-*Updatethenuma-cpulookuptablewiththenewmappings,evenfor-*offlineCPUs.Itisbesttoperformthisupdatefromthestop--*machinecontext.-*/-if(cpus_locked)-stop_machine_cpuslocked(update_lookup_table,&updates[0],-cpumask_of(raw_smp_processor_id()));-else-stop_machine(update_lookup_table,&updates[0],-cpumask_of(raw_smp_processor_id()));--for(ud=&updates[0];ud;ud=ud->next){-unregister_cpu_under_node(ud->cpu,ud->old_nid);-register_cpu_under_node(ud->cpu,ud->new_nid);--dev=get_cpu_device(ud->cpu);-if(dev)-kobject_uevent(&dev->kobj,KOBJ_CHANGE);-cpumask_clear_cpu(ud->cpu,&cpu_associativity_changes_mask);-changed=1;-}--out:-kfree(updates);-returnchanged;+return0;}intarch_update_cpu_topology(void)
prrn_is_enabled() always returns false/0, so handle_rtas_event() can
be simplified and some dead code can be removed. Use machine_is()
instead of #ifdef to run this code only on pseries, and add an
informational ratelimited message that we are ignoring the
events. PRRN events are relatively rare in normal operation and
usually arise from operator-initiated actions such as a DPO (Dynamic
Platform Optimizer) run.
Eventually we do want to consume these events and update the device
tree, but that needs more care to be safe vs LPM and DLPAR.
Signed-off-by: Nathan Lynch <redacted>
---
arch/powerpc/kernel/rtasd.c | 28 +++-------------------------
1 file changed, 3 insertions(+), 25 deletions(-)
@@ -273,37 +273,15 @@ void pSeries_log_error(char *buf, unsigned int err_type, int fatal)}}-#ifdef CONFIG_PPC_PSERIES-staticvoidhandle_prrn_event(s32scope)-{-/*-*ForPRRN,wemustpassthenegativeofthescopevaluein-*theRTASevent.-*/-pseries_devicetree_update(-scope);-numa_update_cpu_topology(false);-}-staticvoidhandle_rtas_event(conststructrtas_error_log*log){-if(rtas_error_type(log)!=RTAS_TYPE_PRRN||!prrn_is_enabled())+if(!machine_is(pseries))return;-/* For PRRN Events the extended log length is used to denote-*thescopeforcallingrtasupdate-nodes.-*/-handle_prrn_event(rtas_error_extended_log_length(log));+if(rtas_error_type(log)==RTAS_TYPE_PRRN)+pr_info_ratelimited("Platform resource reassignment ignored.\n");}-#else--staticvoidhandle_rtas_event(conststructrtas_error_log*log)-{-return;-}--#endif-staticintrtas_log_open(structinode*inode,structfile*file){return0;
pseries_devicetree_update() is no longer called with PRRN_SCOPE. The
purpose of prrn_update_node() was to remove and then add back a LMB
whose NUMA assignment had changed. This has never been reliable, and
this codepath has been default-disabled for several releases. Remove
prrn_update_node().
Signed-off-by: Nathan Lynch <redacted>
---
arch/powerpc/platforms/pseries/mobility.c | 27 -----------------------
1 file changed, 27 deletions(-)
pseries_update_drconf_memory() runs from a DT notifier in response to
an update to the ibm,dynamic-memory property of the
/ibm,dynamic-reconfiguration-memory node. This property is an older
less compact format than the ibm,dynamic-memory-v2 property used in
most currently supported firmwares. There has never been an equivalent
function for the v2 property.
pseries_update_drconf_memory() compares the 'assigned' flag for each
LMB in the old vs new properties and adds or removes the block
accordingly. However it appears to be of no actual utility:
* Partition suspension and PRRNs are specified only to change LMBs'
NUMA affinity information. This notifier should be a no-op for those
scenarios since the assigned flags should not change.
* The memory hotplug/DLPAR path has a hack which short-circuits
execution of the notifier:
dlpar_memory()
...
rtas_hp_event = true;
drmem_update_dt()
of_update_property()
pseries_memory_notifier()
pseries_update_drconf_memory()
if (rtas_hp_event) return;
So this code only makes sense as a relic of the time when more of the
DLPAR workflow took place in user space. I don't see a purpose for it
now.
Signed-off-by: Nathan Lynch <redacted>
---
.../platforms/pseries/hotplug-memory.c | 65 +------------------
1 file changed, 1 insertion(+), 64 deletions(-)
@@ -871,11 +869,8 @@ int dlpar_memory(struct pseries_hp_errorlog *hp_elog)break;}-if(!rc){-rtas_hp_event=true;+if(!rc)rc=drmem_update_dt();-rtas_hp_event=false;-}unlock_device_hotplug();returnrc;
@@ -911,60 +906,6 @@ static int pseries_add_mem_node(struct device_node *np)return(ret<0)?-EINVAL:0;}-staticintpseries_update_drconf_memory(structof_reconfig_data*pr)-{-structof_drconf_cell_v1*new_drmem,*old_drmem;-unsignedlongmemblock_size;-u32entries;-__be32*p;-inti,rc=-EINVAL;--if(rtas_hp_event)-return0;--memblock_size=pseries_memory_block_size();-if(!memblock_size)-return-EINVAL;--if(!pr->old_prop)-return0;--p=(__be32*)pr->old_prop->value;-if(!p)-return-EINVAL;--/* The first int of the property is the number of lmb's described-*bytheproperty.Thisisfollowedbyanarrayofof_drconf_cell-*entries.Getthenumberofentriesandskiptothearrayof-*of_drconf_cell's.-*/-entries=be32_to_cpu(*p++);-old_drmem=(structof_drconf_cell_v1*)p;--p=(__be32*)pr->prop->value;-p++;-new_drmem=(structof_drconf_cell_v1*)p;--for(i=0;i<entries;i++){-if((be32_to_cpu(old_drmem[i].flags)&DRCONF_MEM_ASSIGNED)&&-(!(be32_to_cpu(new_drmem[i].flags)&DRCONF_MEM_ASSIGNED))){-rc=pseries_remove_memblock(-be64_to_cpu(old_drmem[i].base_addr),-memblock_size);-break;-}elseif((!(be32_to_cpu(old_drmem[i].flags)&-DRCONF_MEM_ASSIGNED))&&-(be32_to_cpu(new_drmem[i].flags)&-DRCONF_MEM_ASSIGNED)){-rc=memblock_add(be64_to_cpu(old_drmem[i].base_addr),-memblock_size);-rc=(rc<0)?-EINVAL:0;-break;-}-}-returnrc;-}-staticintpseries_memory_notifier(structnotifier_block*nb,unsignedlongaction,void*data){
@@ -978,10 +919,6 @@ static int pseries_memory_notifier(struct notifier_block *nb,caseOF_RECONFIG_DETACH_NODE:err=pseries_remove_mem_node(rd->dn);break;-caseOF_RECONFIG_UPDATE_PROPERTY:-if(!strcmp(rd->prop->name,"ibm,dynamic-memory"))-err=pseries_update_drconf_memory(rd);-break;}returnnotifier_from_errno(err);}
dlpar_memory() no longer has any callers which pass
PSERIES_HP_ELOG_ACTION_READD. Remove this case and the corresponding
unreachable code.
Signed-off-by: Nathan Lynch <redacted>
---
arch/powerpc/include/asm/rtas.h | 1 -
.../platforms/pseries/hotplug-memory.c | 42 -------------------
2 files changed, 43 deletions(-)
@@ -487,40 +487,6 @@ static int dlpar_memory_remove_by_index(u32 drc_index)returnrc;}-staticintdlpar_memory_readd_by_index(u32drc_index)-{-structdrmem_lmb*lmb;-intlmb_found;-intrc;--pr_info("Attempting to update LMB, drc index %x\n",drc_index);--lmb_found=0;-for_each_drmem_lmb(lmb){-if(lmb->drc_index==drc_index){-lmb_found=1;-rc=dlpar_remove_lmb(lmb);-if(!rc){-rc=dlpar_add_lmb(lmb);-if(rc)-dlpar_release_drc(lmb->drc_index);-}-break;-}-}--if(!lmb_found)-rc=-EINVAL;--if(rc)-pr_info("Failed to update memory at %llx\n",-lmb->base_addr);-else-pr_info("Memory at %llx was updated\n",lmb->base_addr);--returnrc;-}-staticintdlpar_memory_remove_by_ic(u32lmbs_to_remove,u32drc_index){structdrmem_lmb*lmb,*start_lmb,*end_lmb;
@@ -617,10 +583,6 @@ static int dlpar_memory_remove_by_index(u32 drc_index){return-EOPNOTSUPP;}-staticintdlpar_memory_readd_by_index(u32drc_index)-{-return-EOPNOTSUPP;-}staticintdlpar_memory_remove_by_ic(u32lmbs_to_remove,u32drc_index){
Remove the /proc/powerpc/topology_updates interface and the
topology_updates=on/off command line argument. The internal
topology_updates_enabled flag remains for now, but always false.
Looks good to me.
Reviewed-by: Srikar Dronamraju <redacted>
@@ -1632,52 +1612,6 @@ int prrn_is_enabled(void)returnprrn_enabled;}-staticinttopology_read(structseq_file*file,void*v)-{-if(vphn_enabled||prrn_enabled)-seq_puts(file,"on\n");-else-seq_puts(file,"off\n");--return0;-}--staticinttopology_open(structinode*inode,structfile*file)-{-returnsingle_open(file,topology_read,NULL);-}--staticssize_ttopology_write(structfile*file,constchar__user*buf,-size_tcount,loff_t*off)-{-charkbuf[4];/* "on" or "off" plus null. */-intread_len;--read_len=count<3?count:3;-if(copy_from_user(kbuf,buf,read_len))-return-EINVAL;--kbuf[read_len]='\0';--if(!strncmp(kbuf,"on",2)){-topology_updates_enabled=true;-start_topology_update();-}elseif(!strncmp(kbuf,"off",3)){-stop_topology_update();-topology_updates_enabled=false;-}else-return-EINVAL;--returncount;-}--staticconststructproc_opstopology_proc_ops={-.proc_read=seq_read,-.proc_write=topology_write,-.proc_open=topology_open,-.proc_release=single_release,-};-staticinttopology_update_init(void){start_topology_update();
@@ -1685,9 +1619,6 @@ static int topology_update_init(void)if(vphn_enabled)topology_schedule_update();-if(!proc_create("powerpc/topology_updates",0644,NULL,&topology_proc_ops))-return-ENOMEM;-topology_inited=1;return0;}
Since the topology_updates_enabled flag is now always false, remove it
and the code which has become unreachable. This is the minimum change
that prevents 'defined but unused' warnings emitted by the compiler
after stubbing out the start/stop_topology_updates() functions.
Signed-off-by: Nathan Lynch <redacted>
Looks good to me.
Reviewed-by: Srikar Dronamraju <redacted>
--
Thanks and Regards
Srikar Dronamraju
Since vphn_enabled is always 0, we can stub out
timed_topology_update() and remove the code which becomes unreachable.
Signed-off-by: Nathan Lynch <redacted>
---
arch/powerpc/mm/numa.c | 21 ---------------------
1 file changed, 21 deletions(-)
Looks good to me.
Reviewed-by: Srikar Dronamraju <redacted>
--
Thanks and Regards
Srikar Dronamraju
Since vphn_enabled is always 0, we can remove the call to
topology_schedule_update() and remove the code which becomes
unreachable as a result.
Signed-off-by: Nathan Lynch <redacted>
---
arch/powerpc/mm/numa.c | 14 --------------
1 file changed, 14 deletions(-)
Looks good to me.
Reviewed-by: Srikar Dronamraju <redacted>
--
Thanks and Regards
Srikar Dronamraju
These flags are always zero now; remove them and suitably adjust the
remaining references to them.
Signed-off-by: Nathan Lynch <redacted>
---
arch/powerpc/mm/numa.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
Looks good to me.
Reviewed-by: Srikar Dronamraju <redacted>
--
Thanks and Regards
Srikar Dronamraju
Previous changes have removed the code which sets bits in
cpu_associativity_changes_mask and thus it is never modifed at
runtime. From this we can reason that numa_update_cpu_topology()
always returns 0 without doing anything. Remove the body of
numa_update_cpu_topology() and remove all code which becomes
unreachable as a result.
Signed-off-by: Nathan Lynch <redacted>
---
arch/powerpc/mm/numa.c | 193 +----------------------------------------
1 file changed, 1 insertion(+), 192 deletions(-)
Looks good to me.
Reviewed-by: Srikar Dronamraju <redacted>
--
Thanks and Regards
Srikar Dronamraju
prrn_is_enabled() always returns false/0, so handle_rtas_event() can
be simplified and some dead code can be removed. Use machine_is()
instead of #ifdef to run this code only on pseries, and add an
informational ratelimited message that we are ignoring the
events. PRRN events are relatively rare in normal operation and
usually arise from operator-initiated actions such as a DPO (Dynamic
Platform Optimizer) run.
Eventually we do want to consume these events and update the device
tree, but that needs more care to be safe vs LPM and DLPAR.
Signed-off-by: Nathan Lynch <redacted>
---
arch/powerpc/kernel/rtasd.c | 28 +++-------------------------
1 file changed, 3 insertions(+), 25 deletions(-)
Looks good to me.
Reviewed-by: Srikar Dronamraju <redacted>
--
Thanks and Regards
Srikar Dronamraju
From: Michael Ellerman <hidden> Date: 2020-07-16 14:07:34
On Fri, 12 Jun 2020 00:12:20 -0500, Nathan Lynch wrote:
Two major parts to this series:
1. Removal of the extended cede offline mode for CPUs as well as the
partition suspend code which accommodates it by temporarily
onlining all CPUs prior to suspending the LPAR. This solves some
accounting problems, simplifies the pseries CPU hotplug code, and
greatly uncomplicates the existing partition suspend code, easing
a much-needed transition to the Linux suspend framework. The two
patches which make up this part have been posted before:
[...]