Re: /proc/bus/pci and domains

On Wed, Feb 13, 2008 at 04:53:55PM +0100, Martin Mares wrote:

Hi Greg et al.,

I have noticed that lspci with the proc back-end does not work properly
on ppc64. The problem turned out to be a strange hack in drivers/pci/proc.c
causing the following behavior on machines with multiple PCI domains:
The directory names under /proc/bus/pci/ have a domain number added (which
is not backward compatible, but at least consistent), but on the other hand
/proc/bus/pci/devices contains bus numbers without the domains, happily
making multiple entries with the same bus and device number.

Is there any serious reason for this behavior?

I have no idea, it sounds like a PPC specific thing, not anything the
PCI core does, right?  So I'll add the ppc list to the cc: and ask if
anyone there has any ideas?

Rest of the original email follows...

The original /proc/bus/pci/ (as I have designed it years ago) does not
have any means of carrying the domain numbers, so it cannot be solved
in a backward-compatible way, but the inconsistency between the list of
devices and the actual directories leaves me puzzled.

Wouldn't it be better to make the same backward-incompatible change
in /proc/bus/pci/devices, so that at least new programs can use the
thing?

Of course, all this is of minor importance as all new programs know
how to use the sysfs anyway (I have found the problem only because
I forgot to mount sysfs to a chroot), but if we want to keep /proc/bus/pci/,
we should fix it.

				Have a nice fortnight
-- 
Martin `MJ' Mares                          [off-list ref]   http://mj.ucw.cz/
Faculty of Math and Physics, Charles University, Prague, Czech Rep., Earth
Entropy isn't what it used to be.

Hi Greg!

I have no idea, it sounds like a PPC specific thing, not anything the
PCI core does, right?  So I'll add the ppc list to the cc: and ask if
anyone there has any ideas?

Unfortunately, this is not PPC specific at all -- proc_attach_device()
calls pci_proc_domain(bus), which is an arch-specific hook turning on
the weird behavior I described. This hook can return non-zero even on
x86 if there are buses outside domain 0.

I think that the hook itself is correct, it only should trigger more
consistent behavior of the generic code :)

Would it be acceptable for the PPC folks to modify the code to add
domain numbers to the device numbers in /proc/bus/pci/devices in the
same format as the bus directories already have?

If so, I will cook up a patch.

				Have a nice fortnight
-- 
Martin `MJ' Mares                          [off-list ref]   http://mj.ucw.cz/
Faculty of Math and Physics, Charles University, Prague, Czech Rep., Earth
"Dijkstra probably hates me." -- /usr/src/linux/kernel/sched.c

On Thursday, February 14, 2008 1:40 am Martin Mares wrote:

Hi Greg!

quoted

I have no idea, it sounds like a PPC specific thing, not anything the
PCI core does, right?  So I'll add the ppc list to the cc: and ask if
anyone there has any ideas?

Unfortunately, this is not PPC specific at all -- proc_attach_device()
calls pci_proc_domain(bus), which is an arch-specific hook turning on
the weird behavior I described. This hook can return non-zero even on
x86 if there are buses outside domain 0.

I think that the hook itself is correct, it only should trigger more
consistent behavior of the generic code :)

Would it be acceptable for the PPC folks to modify the code to add
domain numbers to the device numbers in /proc/bus/pci/devices in the
same format as the bus directories already have?

If so, I will cook up a patch.

I've run into this too, on machines with many domains.  It's unfortunate that 
there's no way to preserve compatibility, but I agree that the brokenness may 
as well be consistent.

Jesse

From: Martin Mares <mj@ucw.cz>
Date: Thu, 14 Feb 2008 10:40:22 +0100

Would it be acceptable for the PPC folks to modify the code to add
domain numbers to the device numbers in /proc/bus/pci/devices in the
same format as the bus directories already have?

Be careful, things like the X server parse that file, and X's
parser does not expect domain numbers there so you will break
things :-(

Hi!

quoted

Would it be acceptable for the PPC folks to modify the code to add
domain numbers to the device numbers in /proc/bus/pci/devices in the
same format as the bus directories already have?

Be careful, things like the X server parse that file, and X's
parser does not expect domain numbers there so you will break
things :-(

Yes, but neither does it expect domain numbers in the directory names,
I guess :-)

But thanks for the tip, I will check X before I post the patch.

				Have a nice fortnight
-- 
Martin `MJ' Mares                          [off-list ref]   http://mj.ucw.cz/
Faculty of Math and Physics, Charles University, Prague, Czech Rep., Earth
A student who changes the course of history is probably taking an exam.

Hi!

quoted

Be careful, things like the X server parse that file, and X's
parser does not expect domain numbers there so you will break
things :-(

It might be best if we only place domain 0 in that file.

Yes, that's another possilibity. However, in such cases we should probably
create the subdirectories only for domain 0 as well.

				Have a nice fortnight
-- 
Martin `MJ' Mares                          [off-list ref]   http://mj.ucw.cz/
Faculty of Math and Physics, Charles University, Prague, Czech Rep., Earth
"Beware of bugs in the above code; I have only proved it correct, not tried it." -- D.E.K.

On Thu, Feb 14, 2008 at 01:43:05PM -0800, David Miller wrote:

From: Martin Mares <mj@ucw.cz>
Date: Thu, 14 Feb 2008 10:40:22 +0100

quoted

Would it be acceptable for the PPC folks to modify the code to add
domain numbers to the device numbers in /proc/bus/pci/devices in the
same format as the bus directories already have?

Be careful, things like the X server parse that file, and X's
parser does not expect domain numbers there so you will break
things :-(

It might be best if we only place domain 0 in that file.

-- 
Intel are signing my paycheques ... these opinions are still mine
"Bill, look, we understand that you're interested in selling us this
operating system, but compare it to ours.  We can't possibly take such
a retrograde step."

From: Martin Mares <mj@ucw.cz>
Date: Thu, 14 Feb 2008 22:45:14 +0100

But thanks for the tip, I will check X before I post the patch.

X now uses sysfs when available, therefore I think it's
best not to try and muck with this deprecated interface
if possible.

X now uses sysfs when available, therefore I think it's
best not to try and muck with this deprecated interface
if possible.

However, the interface is fatally broken on machines with domains
and the breakage is visible (e.g., run lspci when /sys is not mounted).

I originally intended to just disable the proc backend on the affected
architectures, but it turned out than even x86 can be affected.

				Have a nice fortnight
-- 
Martin `MJ' Mares                          [off-list ref]   http://mj.ucw.cz/
Faculty of Math and Physics, Charles University, Prague, Czech Rep., Earth
"God doesn't play dice."   -- Albert Einstein