Re: vio.c:__vio_register_driver && LPAR Migration issue

On 08/03/2018 02:23 PM, Tyrel Datwyler wrote:

On 08/02/2018 11:15 AM, Michael Bringmann wrote:

quoted

Hello:
    I have been observing an anomaly during LPAR migrations between
a couple of P8 systems.

This is the problem.  After migrating an LPAR, the PPC mobility code
receives RTAS requests to delete nodes with platform-/hardware-specific
attributes when restarting the kernel after a migration.  My example is
for migration between a P8 Alpine and a P8 Brazos.  Among the nodes
that I see being deleted are 'ibm,random-v1', 'ibm,compression-v1',
'ibm,platform-facilities', and 'ibm,sym-encryption-v1'.  Of these
nodes, the following are created during initial boot by calls to
vio_register_driver:

drivers/char/hw_random/pseries-rng.c
        ibm,random-v1

drivers/crypto/nx/nx-842-pseries.c
        ibm,compression-v1

drivers/crypto/nx/nx.c
        ibm,sym-encryption-v1

After the migration, these nodes are deleted, but nothing recreates
them.  If I boot the LPAR on the target system, the nodes are added
again.

My question is how do we recreate these nodes after migration?

Hmm, I'd have to see the scenario in action, but these should be added back by ibm,update-nodes RTAS call. There is some debug code in driver/of/dynamic.c that can be enabled that will log node/property dynamic reconfiguration events.

I took a quick look by turning of phandle caching and turning on the OF reconfig debug on one of my lpars. I noticed that it looks like this could be a firmware issue. Looks to me like we aren't being notified to re-add those child nodes of ibm,platform-facilities.

[ 1671.638041] OF: notify DETACH_NODE     /cpus/l2-cache@2022
[ 1671.638094] OF: notify DETACH_NODE     /cpus/l3-cache@3122
[ 1671.638119] OF: notify DETACH_NODE     /cpus/l2-cache@2023
[ 1671.638136] OF: notify DETACH_NODE     /cpus/l3-cache@3123
[ 1671.638164] OF: notify DETACH_NODE     /ibm,platform-facilities/ibm,random-v1
[ 1671.638182] OF: notify DETACH_NODE     /ibm,platform-facilities/ibm,compression-v1
[ 1671.638198] OF: notify DETACH_NODE     /ibm,platform-facilities/ibm,sym-encryption-v1
[ 1671.638219] OF: notify DETACH_NODE     /ibm,platform-facilities
[ 1671.776419] OF: notify UPDATE_PROPERTY /:ibm,model-class

... snipped property updates

[ 1672.129941] OF: notify UPDATE_PROPERTY /cpus/PowerPC,POWER8@8:ibm,dfp
[ 1672.147551] OF: notify ATTACH_NODE     /cpus/l2-cache@202e
[ 1672.166321] OF: notify ATTACH_NODE     /cpus/l3-cache@312e
[ 1672.183971] OF: notify ATTACH_NODE     /cpus/l2-cache@202f
[ 1672.202752] OF: notify ATTACH_NODE     /cpus/l3-cache@312f
[ 1672.230760] OF: notify ATTACH_NODE     /ibm,platform-facilities

Need to verify this by tracing the RTAS calls to ibm,update-nodes. I'll try and look at that tomorrow. The loop that processes the nodes to update in pseries_devicetree_update() blindly ignores the return codes from delete_dt_node(), update_dt_node(), and add_dt_node(). So, it is also possible that we are notified, but are silently failing the add.

-Tyrel