From: Scott Wood <hidden> Date: 2015-10-21 05:54:29
On Mon, 2015-09-21 at 16:03 +0200, Tomeu Vizoso wrote:
Instead of trying to match and probe platform and AMBA devices right
after each is registered, delay their probes until device_initcall_sync.
This means that devices will start probing once all built-in drivers
have registered, and after all platform and AMBA devices from the DT
have been registered already.
This allows us to prevent deferred probes by probing dependencies on
demand.
Signed-off-by: Tomeu Vizoso <redacted>
---
Changes in v4:
- Also defer probes of AMBA devices registered from the DT as they can
also request resources.
drivers/of/platform.c | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
This breaks arch/powerpc/sysdev/fsl_pci.c. The PCI bus is an OF platform
device, and it must be probed before pcibios_init() which is a
subsys_initcall(), or else the PCI bus never gets scanned.
-Scott
From: Rob Herring <robh+dt@kernel.org> Date: 2015-10-21 13:45:12
On Wed, Oct 21, 2015 at 12:54 AM, Scott Wood [off-list ref] wrote:
On Mon, 2015-09-21 at 16:03 +0200, Tomeu Vizoso wrote:
quoted
Instead of trying to match and probe platform and AMBA devices right
after each is registered, delay their probes until device_initcall_sync.
This means that devices will start probing once all built-in drivers
have registered, and after all platform and AMBA devices from the DT
have been registered already.
This allows us to prevent deferred probes by probing dependencies on
demand.
Signed-off-by: Tomeu Vizoso <redacted>
---
Changes in v4:
- Also defer probes of AMBA devices registered from the DT as they can
also request resources.
drivers/of/platform.c | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
This breaks arch/powerpc/sysdev/fsl_pci.c. The PCI bus is an OF platform
device, and it must be probed before pcibios_init() which is a
subsys_initcall(), or else the PCI bus never gets scanned.
Thanks for the report. This is probably getting dropped, but it could
be disabled for PPC.
Any plans to fix this and make PCI hosts hotplugable? For the scanning
part, generally the host controller drivers are responsible for
scanning their bus now.
Rob
From: Scott Wood <hidden> Date: 2015-10-21 22:52:11
On Wed, 2015-10-21 at 08:44 -0500, Rob Herring wrote:
On Wed, Oct 21, 2015 at 12:54 AM, Scott Wood [off-list ref]
wrote:
quoted
On Mon, 2015-09-21 at 16:03 +0200, Tomeu Vizoso wrote:
quoted
Instead of trying to match and probe platform and AMBA devices right
after each is registered, delay their probes until device_initcall_sync.
This means that devices will start probing once all built-in drivers
have registered, and after all platform and AMBA devices from the DT
have been registered already.
This allows us to prevent deferred probes by probing dependencies on
demand.
Signed-off-by: Tomeu Vizoso <redacted>
---
Changes in v4:
- Also defer probes of AMBA devices registered from the DT as they can
also request resources.
drivers/of/platform.c | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
This breaks arch/powerpc/sysdev/fsl_pci.c. The PCI bus is an OF platform
device, and it must be probed before pcibios_init() which is a
subsys_initcall(), or else the PCI bus never gets scanned.
Thanks for the report. This is probably getting dropped, but it could
be disabled for PPC.
I don't think that adding another arbitrary arch difference would be the
right solution.
Any plans to fix this and make PCI hosts hotplugable? For the scanning
part, generally the host controller drivers are responsible for
scanning their bus now.
Scanning from the host controller driver seems like a reasonable goal, though
it'd take a bit of digging to extract whatever other things fsl_pci may
depend on from the common PPC PCI code, in particular the various things that
pcibios_resource_survey() does after all PCI buses have been scanned.
There's also check_swiotlb_enabled(), another subsys_initcall, which frees
the swiotlb memory if ppc_swiotlb_enable hasn't been set. The PCI host
controller probe sets ppc_swiotlb_enable if it wasn't able to create an
inbound mapping for all RAM. Even if we were to change that to a later
initcall, there's nothing later than late_initcall that we could use.
-Scott
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2015-10-22 00:34:52
On Wed, 2015-10-21 at 00:54 -0500, Scott Wood wrote:
On Mon, 2015-09-21 at 16:03 +0200, Tomeu Vizoso wrote:
quoted
Instead of trying to match and probe platform and AMBA devices right
after each is registered, delay their probes until device_initcall_sync.
This means that devices will start probing once all built-in drivers
have registered, and after all platform and AMBA devices from the DT
have been registered already.
This allows us to prevent deferred probes by probing dependencies on
demand.
Signed-off-by: Tomeu Vizoso <redacted>
---
Changes in v4:
- Also defer probes of AMBA devices registered from the DT as they can
also request resources.
drivers/of/platform.c | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
This breaks arch/powerpc/sysdev/fsl_pci.c. The PCI bus is an OF platform
device, and it must be probed before pcibios_init() which is a
subsys_initcall(), or else the PCI bus never gets scanned.
Ah right. This is presumably why I'm not seeing any PCI devices on my p5020ds
with linux-next.
cheers
On 22 October 2015 at 00:51, Scott Wood [off-list ref] wrote:
On Wed, 2015-10-21 at 08:44 -0500, Rob Herring wrote:
quoted
On Wed, Oct 21, 2015 at 12:54 AM, Scott Wood [off-list ref]
wrote:
quoted
On Mon, 2015-09-21 at 16:03 +0200, Tomeu Vizoso wrote:
quoted
Instead of trying to match and probe platform and AMBA devices right
after each is registered, delay their probes until device_initcall_sync.
This means that devices will start probing once all built-in drivers
have registered, and after all platform and AMBA devices from the DT
have been registered already.
This allows us to prevent deferred probes by probing dependencies on
demand.
Signed-off-by: Tomeu Vizoso <redacted>
---
Changes in v4:
- Also defer probes of AMBA devices registered from the DT as they can
also request resources.
drivers/of/platform.c | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
This breaks arch/powerpc/sysdev/fsl_pci.c. The PCI bus is an OF platform
device, and it must be probed before pcibios_init() which is a
subsys_initcall(), or else the PCI bus never gets scanned.
Thanks for the report. This is probably getting dropped, but it could
be disabled for PPC.
I don't think that adding another arbitrary arch difference would be the
right solution.
I think Rob meant temporarily disable it while things get fixed. At
least, I don't see any reason why PPC wouldn't benefit from this
series.
Regards,
Tomeu
quoted
Any plans to fix this and make PCI hosts hotplugable? For the scanning
part, generally the host controller drivers are responsible for
scanning their bus now.
Scanning from the host controller driver seems like a reasonable goal, though
it'd take a bit of digging to extract whatever other things fsl_pci may
depend on from the common PPC PCI code, in particular the various things that
pcibios_resource_survey() does after all PCI buses have been scanned.
There's also check_swiotlb_enabled(), another subsys_initcall, which frees
the swiotlb memory if ppc_swiotlb_enable hasn't been set. The PCI host
controller probe sets ppc_swiotlb_enable if it wasn't able to create an
inbound mapping for all RAM. Even if we were to change that to a later
initcall, there's nothing later than late_initcall that we could use.
-Scott
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Scott Wood <hidden> Date: 2015-10-22 21:27:29
On Thu, 2015-10-22 at 15:04 +0200, Tomeu Vizoso wrote:
On 22 October 2015 at 00:51, Scott Wood [off-list ref] wrote:
quoted
On Wed, 2015-10-21 at 08:44 -0500, Rob Herring wrote:
quoted
On Wed, Oct 21, 2015 at 12:54 AM, Scott Wood [off-list ref]
wrote:
quoted
On Mon, 2015-09-21 at 16:03 +0200, Tomeu Vizoso wrote:
quoted
Instead of trying to match and probe platform and AMBA devices right
after each is registered, delay their probes until
device_initcall_sync.
This means that devices will start probing once all built-in drivers
have registered, and after all platform and AMBA devices from the DT
have been registered already.
This allows us to prevent deferred probes by probing dependencies on
demand.
Signed-off-by: Tomeu Vizoso <redacted>
---
Changes in v4:
- Also defer probes of AMBA devices registered from the DT as they
can
also request resources.
drivers/of/platform.c | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
This breaks arch/powerpc/sysdev/fsl_pci.c. The PCI bus is an OF
platform
device, and it must be probed before pcibios_init() which is a
subsys_initcall(), or else the PCI bus never gets scanned.
Thanks for the report. This is probably getting dropped, but it could
be disabled for PPC.
I don't think that adding another arbitrary arch difference would be the
right solution.
I think Rob meant temporarily disable it while things get fixed. At
least,
So, what is the permanent fix for the swiotlb issue (or more generally, the
inability to have a late_initcall that runs after non-module, non-hotplug
platform devices have been probed)?
I don't see any reason why PPC wouldn't benefit from this
series.
It's not clear to me what the benefit of this is at all, much less for PPC.
What is the fundamental problem with deferred probes? In the cover letter
you say this change saves 2.3 seconds, but where is that time being consumed?
Are the drivers taking too long in their probe function trying to initialize
and then deferring, rather than checking for dependencies up front? Or are
there really so many devices and such a pessimal ordering that most of the
time is spent iterating through and reordering the list, with each defer
happening quickly?
Even if something different does need to be done at this level, forcing all
OF platform devices to be probed at the late_initcall level seems quite
intrusive. You limited it to OF because people complained that other things
will break. Things still broke. Surely there's a better way to address the
problem. Can't the delay be requested by drivers that might otherwise need
to defer (which could be done incrementally, focusing on the worst
performance problems), rather than enabling it for everything?
-Scott
From: Rafael J. Wysocki <hidden> Date: 2015-10-24 13:22:56
On Thursday, October 22, 2015 04:27:10 PM Scott Wood wrote:
On Thu, 2015-10-22 at 15:04 +0200, Tomeu Vizoso wrote:
quoted
On 22 October 2015 at 00:51, Scott Wood [off-list ref] wrote:
quoted
On Wed, 2015-10-21 at 08:44 -0500, Rob Herring wrote:
quoted
On Wed, Oct 21, 2015 at 12:54 AM, Scott Wood [off-list ref]
wrote:
quoted
On Mon, 2015-09-21 at 16:03 +0200, Tomeu Vizoso wrote:
quoted
Instead of trying to match and probe platform and AMBA devices right
after each is registered, delay their probes until
device_initcall_sync.
This means that devices will start probing once all built-in drivers
have registered, and after all platform and AMBA devices from the DT
have been registered already.
This allows us to prevent deferred probes by probing dependencies on
demand.
Signed-off-by: Tomeu Vizoso <redacted>
---
Changes in v4:
- Also defer probes of AMBA devices registered from the DT as they
can
also request resources.
drivers/of/platform.c | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
This breaks arch/powerpc/sysdev/fsl_pci.c. The PCI bus is an OF
platform
device, and it must be probed before pcibios_init() which is a
subsys_initcall(), or else the PCI bus never gets scanned.
Thanks for the report. This is probably getting dropped, but it could
be disabled for PPC.
I don't think that adding another arbitrary arch difference would be the
right solution.
I think Rob meant temporarily disable it while things get fixed. At
least,
So, what is the permanent fix for the swiotlb issue (or more generally, the
inability to have a late_initcall that runs after non-module, non-hotplug
platform devices have been probed)?
quoted
I don't see any reason why PPC wouldn't benefit from this
series.
It's not clear to me what the benefit of this is at all, much less for PPC.
What is the fundamental problem with deferred probes? In the cover letter
you say this change saves 2.3 seconds, but where is that time being consumed?
Are the drivers taking too long in their probe function trying to initialize
and then deferring, rather than checking for dependencies up front? Or are
there really so many devices and such a pessimal ordering that most of the
time is spent iterating through and reordering the list, with each defer
happening quickly?
Even if something different does need to be done at this level, forcing all
OF platform devices to be probed at the late_initcall level seems quite
intrusive.
Totally agreed.
You limited it to OF because people complained that other things
will break.
Right.
And I'm not sure why that was regarded as a good enough reason to do it.
Things still broke.
Yes, they did.
Surely there's a better way to address the
problem. Can't the delay be requested by drivers that might otherwise need
to defer (which could be done incrementally, focusing on the worst
performance problems), rather than enabling it for everything?
Well, I was suggesting to use an opt-in flag there, but I'm not sure if Tomeu
took that into consideration.
In any case, probing is just one aspect of a deeper issue, which is that
we have no way to represent functional dependencies between devices.
Thanks,
Rafael
On 22 October 2015 at 23:27, Scott Wood [off-list ref] wrote:
On Thu, 2015-10-22 at 15:04 +0200, Tomeu Vizoso wrote:
quoted
On 22 October 2015 at 00:51, Scott Wood [off-list ref] wrote:
quoted
On Wed, 2015-10-21 at 08:44 -0500, Rob Herring wrote:
quoted
On Wed, Oct 21, 2015 at 12:54 AM, Scott Wood [off-list ref]
wrote:
quoted
On Mon, 2015-09-21 at 16:03 +0200, Tomeu Vizoso wrote:
quoted
Instead of trying to match and probe platform and AMBA devices right
after each is registered, delay their probes until
device_initcall_sync.
This means that devices will start probing once all built-in drivers
have registered, and after all platform and AMBA devices from the DT
have been registered already.
This allows us to prevent deferred probes by probing dependencies on
demand.
Signed-off-by: Tomeu Vizoso <redacted>
---
Changes in v4:
- Also defer probes of AMBA devices registered from the DT as they
can
also request resources.
drivers/of/platform.c | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
This breaks arch/powerpc/sysdev/fsl_pci.c. The PCI bus is an OF
platform
device, and it must be probed before pcibios_init() which is a
subsys_initcall(), or else the PCI bus never gets scanned.
Thanks for the report. This is probably getting dropped, but it could
be disabled for PPC.
I don't think that adding another arbitrary arch difference would be the
right solution.
I think Rob meant temporarily disable it while things get fixed. At
least,
So, what is the permanent fix for the swiotlb issue (or more generally, the
inability to have a late_initcall that runs after non-module, non-hotplug
platform devices have been probed)?
If the code in pcibios_init() depends on the PCI bus device having
probed, then I would recommend making that dependency explicit by
calling of_device_probe() on the OF node of the PCI controller when
looking it up.
quoted
I don't see any reason why PPC wouldn't benefit from this
series.
It's not clear to me what the benefit of this is at all, much less for PPC.
What is the fundamental problem with deferred probes? In the cover letter
you say this change saves 2.3 seconds, but where is that time being consumed?
Are the drivers taking too long in their probe function trying to initialize
and then deferring, rather than checking for dependencies up front? Or are
there really so many devices and such a pessimal ordering that most of the
time is spent iterating through and reordering the list, with each defer
happening quickly?
The problem is that a device that defers its probe is currently sent
to the back of the queue, and that's undesired in some use cases in
which there's a device that should be up as soon as possible during
boot (and boot takes a long time). So the goal is to change the order
in which devices with dependencies end up probing.
Even if something different does need to be done at this level, forcing all
OF platform devices to be probed at the late_initcall level seems quite
intrusive. You limited it to OF because people complained that other things
will break. Things still broke. Surely there's a better way to address the
problem. Can't the delay be requested by drivers that might otherwise need
to defer (which could be done incrementally, focusing on the worst
performance problems), rather than enabling it for everything?
Yes, given the amount of breakage that's a sensible option. But in any
case and even if this series is most likely to be dropped, I recommend
to make explicit as many implicit dependencies as possible.
Regards,
Tomeu
From: Rob Herring <robh+dt@kernel.org> Date: 2015-10-29 04:18:22
On Wed, Oct 28, 2015 at 9:40 AM, Tomeu Vizoso
[off-list ref] wrote:
On 22 October 2015 at 23:27, Scott Wood [off-list ref] wrote:
quoted
On Thu, 2015-10-22 at 15:04 +0200, Tomeu Vizoso wrote:
quoted
On 22 October 2015 at 00:51, Scott Wood [off-list ref] wrote:
quoted
On Wed, 2015-10-21 at 08:44 -0500, Rob Herring wrote:
quoted
On Wed, Oct 21, 2015 at 12:54 AM, Scott Wood [off-list ref]
wrote:
quoted
On Mon, 2015-09-21 at 16:03 +0200, Tomeu Vizoso wrote:
quoted
Instead of trying to match and probe platform and AMBA devices right
after each is registered, delay their probes until
device_initcall_sync.
This means that devices will start probing once all built-in drivers
have registered, and after all platform and AMBA devices from the DT
have been registered already.
This allows us to prevent deferred probes by probing dependencies on
demand.
Signed-off-by: Tomeu Vizoso <redacted>
---
Changes in v4:
- Also defer probes of AMBA devices registered from the DT as they
can
also request resources.
drivers/of/platform.c | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
This breaks arch/powerpc/sysdev/fsl_pci.c. The PCI bus is an OF
platform
device, and it must be probed before pcibios_init() which is a
subsys_initcall(), or else the PCI bus never gets scanned.
Thanks for the report. This is probably getting dropped, but it could
be disabled for PPC.
I don't think that adding another arbitrary arch difference would be the
right solution.
I think Rob meant temporarily disable it while things get fixed. At
least,
So, what is the permanent fix for the swiotlb issue (or more generally, the
inability to have a late_initcall that runs after non-module, non-hotplug
platform devices have been probed)?
If the code in pcibios_init() depends on the PCI bus device having
probed, then I would recommend making that dependency explicit by
calling of_device_probe() on the OF node of the PCI controller when
looking it up.
quoted
quoted
I don't see any reason why PPC wouldn't benefit from this
series.
It's not clear to me what the benefit of this is at all, much less for PPC.
What is the fundamental problem with deferred probes? In the cover letter
you say this change saves 2.3 seconds, but where is that time being consumed?
Are the drivers taking too long in their probe function trying to initialize
and then deferring, rather than checking for dependencies up front? Or are
there really so many devices and such a pessimal ordering that most of the
time is spent iterating through and reordering the list, with each defer
happening quickly?
The problem is that a device that defers its probe is currently sent
to the back of the queue, and that's undesired in some use cases in
which there's a device that should be up as soon as possible during
boot (and boot takes a long time). So the goal is to change the order
in which devices with dependencies end up probing.
quoted
Even if something different does need to be done at this level, forcing all
OF platform devices to be probed at the late_initcall level seems quite
intrusive. You limited it to OF because people complained that other things
will break. Things still broke. Surely there's a better way to address the
problem. Can't the delay be requested by drivers that might otherwise need
to defer (which could be done incrementally, focusing on the worst
performance problems), rather than enabling it for everything?
Yes, given the amount of breakage that's a sensible option. But in any
case and even if this series is most likely to be dropped, I recommend
to make explicit as many implicit dependencies as possible.
It is dropped for now, but I expect much of this to still integrate
with Rafael's proposal. However, it will need to be opt-in it seems.
That is somewhat unfortunate because things like async probe are
opt-in and seem to rarely get used. As to how we can make it opt-in, I
think we can make of_platform_populate() callable multiple times at
the root level. This will allow a platform to skip calling it and then
a late call to it can register the devices after drivers have
registered. This late call will then be a nop for platforms that don't
opt-in and call of_platform_populate themselves.
Rob
From: Scott Wood <hidden> Date: 2015-10-29 16:06:34
On Wed, 2015-10-28 at 15:40 +0100, Tomeu Vizoso wrote:
On 22 October 2015 at 23:27, Scott Wood [off-list ref] wrote:
quoted
On Thu, 2015-10-22 at 15:04 +0200, Tomeu Vizoso wrote:
quoted
On 22 October 2015 at 00:51, Scott Wood [off-list ref] wrote:
quoted
On Wed, 2015-10-21 at 08:44 -0500, Rob Herring wrote:
quoted
On Wed, Oct 21, 2015 at 12:54 AM, Scott Wood <
scottwood@freescale.com>
wrote:
quoted
On Mon, 2015-09-21 at 16:03 +0200, Tomeu Vizoso wrote:
quoted
Instead of trying to match and probe platform and AMBA devices
right
after each is registered, delay their probes until
device_initcall_sync.
This means that devices will start probing once all built-in
drivers
have registered, and after all platform and AMBA devices from
the DT
have been registered already.
This allows us to prevent deferred probes by probing
dependencies on
demand.
Signed-off-by: Tomeu Vizoso <redacted>
---
Changes in v4:
- Also defer probes of AMBA devices registered from the DT as
they
can
also request resources.
drivers/of/platform.c | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
This breaks arch/powerpc/sysdev/fsl_pci.c. The PCI bus is an OF
platform
device, and it must be probed before pcibios_init() which is a
subsys_initcall(), or else the PCI bus never gets scanned.
Thanks for the report. This is probably getting dropped, but it
could
be disabled for PPC.
I don't think that adding another arbitrary arch difference would be
the
right solution.
I think Rob meant temporarily disable it while things get fixed. At
least,
So, what is the permanent fix for the swiotlb issue (or more generally,
the
inability to have a late_initcall that runs after non-module, non-hotplug
platform devices have been probed)?
If the code in pcibios_init() depends on the PCI bus device having
probed, then I would recommend making that dependency explicit by
calling of_device_probe() on the OF node of the PCI controller when
looking it up.
"when looking it up"? pcibios_init() doesn't do anything with the OF node or
know any details about the particular PCI bus host implementation.
quoted
quoted
I don't see any reason why PPC wouldn't benefit from this
series.
It's not clear to me what the benefit of this is at all, much less for
PPC.
What is the fundamental problem with deferred probes? In the cover letter
you say this change saves 2.3 seconds, but where is that time being
consumed?
Are the drivers taking too long in their probe function trying to
initialize
and then deferring, rather than checking for dependencies up front? Or
are
there really so many devices and such a pessimal ordering that most of the
time is spent iterating through and reordering the list, with each defer
happening quickly?
The problem is that a device that defers its probe is currently sent
to the back of the queue, and that's undesired in some use cases in
which there's a device that should be up as soon as possible during
boot (and boot takes a long time). So the goal is to change the order
in which devices with dependencies end up probing.
That doesn't answer my question about where the time is being spent. Did you
profile?
-Scott