This is intentional. The SerDes is attached to the XPCS node because
on RK3568, a single SerDes serves all four XPCS MII ports in QSGMII
mode.
This is not the only device supporting QSGMII, and so the issues you
are addressing should be common to many QSGMII implementations.
Please take a step back. What would a generic solution look like?
Can the reference counting be placed into the core somewhere? The
common clock framework allows a clock to be enabled and disabled by
multiple consumers, and the core clock code does the reference
counting, only calling into the clock driver when state change is
actually needed. Maybe the generic PHY core needs something similar?
Andrew