From: Michael Bringmann <hidden> Date: 2018-08-02 18:15:42
Hello:
I have been observing an anomaly during LPAR migrations between
a couple of P8 systems.
This is the problem. After migrating an LPAR, the PPC mobility code
receives RTAS requests to delete nodes with platform-/hardware-specific
attributes when restarting the kernel after a migration. My example is
for migration between a P8 Alpine and a P8 Brazos. Among the nodes
that I see being deleted are 'ibm,random-v1', 'ibm,compression-v1',
'ibm,platform-facilities', and 'ibm,sym-encryption-v1'. Of these
nodes, the following are created during initial boot by calls to
vio_register_driver:
drivers/char/hw_random/pseries-rng.c
ibm,random-v1
drivers/crypto/nx/nx-842-pseries.c
ibm,compression-v1
drivers/crypto/nx/nx.c
ibm,sym-encryption-v1
After the migration, these nodes are deleted, but nothing recreates
them. If I boot the LPAR on the target system, the nodes are added
again.
My question is how do we recreate these nodes after migration?
Thanks.
--
Michael W. Bringmann
Linux Technology Center
IBM Corporation
Tie-Line 363-5196
External: (512) 286-5196
Cell: (512) 466-0650
mwb@linux.vnet.ibm.com
Hello:
I have been observing an anomaly during LPAR migrations between
a couple of P8 systems.
This is the problem. After migrating an LPAR, the PPC mobility code
receives RTAS requests to delete nodes with platform-/hardware-specific
attributes when restarting the kernel after a migration. My example is
for migration between a P8 Alpine and a P8 Brazos. Among the nodes
that I see being deleted are 'ibm,random-v1', 'ibm,compression-v1',
'ibm,platform-facilities', and 'ibm,sym-encryption-v1'. Of these
nodes, the following are created during initial boot by calls to
vio_register_driver:
drivers/char/hw_random/pseries-rng.c
ibm,random-v1
drivers/crypto/nx/nx-842-pseries.c
ibm,compression-v1
drivers/crypto/nx/nx.c
ibm,sym-encryption-v1
After the migration, these nodes are deleted, but nothing recreates
them. If I boot the LPAR on the target system, the nodes are added
again.
My question is how do we recreate these nodes after migration?
Hmm, I'd have to see the scenario in action, but these should be added back by ibm,update-nodes RTAS call. There is some debug code in driver/of/dynamic.c that can be enabled that will log node/property dynamic reconfiguration events.
-Tyrel
Hello:
I have been observing an anomaly during LPAR migrations between
a couple of P8 systems.
This is the problem. After migrating an LPAR, the PPC mobility code
receives RTAS requests to delete nodes with platform-/hardware-specific
attributes when restarting the kernel after a migration. My example is
for migration between a P8 Alpine and a P8 Brazos. Among the nodes
that I see being deleted are 'ibm,random-v1', 'ibm,compression-v1',
'ibm,platform-facilities', and 'ibm,sym-encryption-v1'. Of these
nodes, the following are created during initial boot by calls to
vio_register_driver:
drivers/char/hw_random/pseries-rng.c
ibm,random-v1
drivers/crypto/nx/nx-842-pseries.c
ibm,compression-v1
drivers/crypto/nx/nx.c
ibm,sym-encryption-v1
After the migration, these nodes are deleted, but nothing recreates
them. If I boot the LPAR on the target system, the nodes are added
again.
My question is how do we recreate these nodes after migration?
Hmm, I'd have to see the scenario in action, but these should be added back by ibm,update-nodes RTAS call. There is some debug code in driver/of/dynamic.c that can be enabled that will log node/property dynamic reconfiguration events.
I took a quick look by turning of phandle caching and turning on the OF reconfig debug on one of my lpars. I noticed that it looks like this could be a firmware issue. Looks to me like we aren't being notified to re-add those child nodes of ibm,platform-facilities.
[ 1671.638041] OF: notify DETACH_NODE /cpus/l2-cache@2022
[ 1671.638094] OF: notify DETACH_NODE /cpus/l3-cache@3122
[ 1671.638119] OF: notify DETACH_NODE /cpus/l2-cache@2023
[ 1671.638136] OF: notify DETACH_NODE /cpus/l3-cache@3123
[ 1671.638164] OF: notify DETACH_NODE /ibm,platform-facilities/ibm,random-v1
[ 1671.638182] OF: notify DETACH_NODE /ibm,platform-facilities/ibm,compression-v1
[ 1671.638198] OF: notify DETACH_NODE /ibm,platform-facilities/ibm,sym-encryption-v1
[ 1671.638219] OF: notify DETACH_NODE /ibm,platform-facilities
[ 1671.776419] OF: notify UPDATE_PROPERTY /:ibm,model-class
... snipped property updates
[ 1672.129941] OF: notify UPDATE_PROPERTY /cpus/PowerPC,POWER8@8:ibm,dfp
[ 1672.147551] OF: notify ATTACH_NODE /cpus/l2-cache@202e
[ 1672.166321] OF: notify ATTACH_NODE /cpus/l3-cache@312e
[ 1672.183971] OF: notify ATTACH_NODE /cpus/l2-cache@202f
[ 1672.202752] OF: notify ATTACH_NODE /cpus/l3-cache@312f
[ 1672.230760] OF: notify ATTACH_NODE /ibm,platform-facilities
Need to verify this by tracing the RTAS calls to ibm,update-nodes. I'll try and look at that tomorrow. The loop that processes the nodes to update in pseries_devicetree_update() blindly ignores the return codes from delete_dt_node(), update_dt_node(), and add_dt_node(). So, it is also possible that we are notified, but are silently failing the add.
-Tyrel
From: Michael Bringmann <hidden> Date: 2018-08-06 15:35:22
On 08/04/2018 06:01 PM, Tyrel Datwyler wrote:
On 08/03/2018 02:23 PM, Tyrel Datwyler wrote:
quoted
On 08/02/2018 11:15 AM, Michael Bringmann wrote:
quoted
Hello:
I have been observing an anomaly during LPAR migrations between
a couple of P8 systems.
This is the problem. After migrating an LPAR, the PPC mobility code
receives RTAS requests to delete nodes with platform-/hardware-specific
attributes when restarting the kernel after a migration. My example is
for migration between a P8 Alpine and a P8 Brazos. Among the nodes
that I see being deleted are 'ibm,random-v1', 'ibm,compression-v1',
'ibm,platform-facilities', and 'ibm,sym-encryption-v1'. Of these
nodes, the following are created during initial boot by calls to
vio_register_driver:
drivers/char/hw_random/pseries-rng.c
ibm,random-v1
drivers/crypto/nx/nx-842-pseries.c
ibm,compression-v1
drivers/crypto/nx/nx.c
ibm,sym-encryption-v1
After the migration, these nodes are deleted, but nothing recreates
them. If I boot the LPAR on the target system, the nodes are added
again.
My question is how do we recreate these nodes after migration?
Hmm, I'd have to see the scenario in action, but these should be added back by ibm,update-nodes RTAS call. There is some debug code in driver/of/dynamic.c that can be enabled that will log node/property dynamic reconfiguration events.
I took a quick look by turning of phandle caching and turning on the OF reconfig debug on one of my lpars. I noticed that it looks like this could be a firmware issue. Looks to me like we aren't being notified to re-add those child nodes of ibm,platform-facilities.
[ 1671.638041] OF: notify DETACH_NODE /cpus/l2-cache@2022
[ 1671.638094] OF: notify DETACH_NODE /cpus/l3-cache@3122
[ 1671.638119] OF: notify DETACH_NODE /cpus/l2-cache@2023
[ 1671.638136] OF: notify DETACH_NODE /cpus/l3-cache@3123
[ 1671.638164] OF: notify DETACH_NODE /ibm,platform-facilities/ibm,random-v1
[ 1671.638182] OF: notify DETACH_NODE /ibm,platform-facilities/ibm,compression-v1
[ 1671.638198] OF: notify DETACH_NODE /ibm,platform-facilities/ibm,sym-encryption-v1
[ 1671.638219] OF: notify DETACH_NODE /ibm,platform-facilities
[ 1671.776419] OF: notify UPDATE_PROPERTY /:ibm,model-class
... snipped property updates
[ 1672.129941] OF: notify UPDATE_PROPERTY /cpus/PowerPC,POWER8@8:ibm,dfp
[ 1672.147551] OF: notify ATTACH_NODE /cpus/l2-cache@202e
[ 1672.166321] OF: notify ATTACH_NODE /cpus/l3-cache@312e
[ 1672.183971] OF: notify ATTACH_NODE /cpus/l2-cache@202f
[ 1672.202752] OF: notify ATTACH_NODE /cpus/l3-cache@312f
[ 1672.230760] OF: notify ATTACH_NODE /ibm,platform-facilities
Need to verify this by tracing the RTAS calls to ibm,update-nodes. I'll try and look at that tomorrow. The loop that processes the nodes to update in pseries_devicetree_update() blindly ignores the return codes from delete_dt_node(), update_dt_node(), and add_dt_node(). So, it is also possible that we are notified, but are silently failing the add.
I am seeing the same set of detach/attach events for migration between an Alpine and Brazos.
I did the logging up in delete_dt_node, update_dt_node, and add_dt_node, and did not see any
notification of additional nodes.
-Tyrel
Michael
--
Michael W. Bringmann
Linux Technology Center
IBM Corporation
Tie-Line 363-5196
External: (512) 286-5196
Cell: (512) 466-0650
mwb@linux.vnet.ibm.com