Re: __ioremap_at() in 2.4.0-test9-pre2

On Wed, Sep 20, 2000 at 09:58:07AM +1100, Paul Mackerras wrote:

Matt Porter writes:

quoted

On Tue, Sep 19, 2000 at 02:59:02PM +1100, Paul Mackerras wrote:

quoted

What I am intending to do is to map the I/O space of all the PCI host
bridges in consecutive areas beginning at some address such as
0xff000000, with some amount of space such as 64kB or 1MB per bridge,
whatever is appropriate.  Then we adjust the I/O port numbers in the
pci_dev structures by adding on host_bridge_nr * space_per_bridge.  As
a side effect, isa_io_base (which is really pci_io_base) becomes a
constant.

Specifically, what problem are you trying to solve with this
implementation?  I gather that we're talking about the legacy I/O
problem that the kernel has.  What are the cases where you need
to use in*/out* calls targetting devices on host bridges other
than the "primary" one?

This is the situation: you have a machine with 2 or more PCI host
bridges.  You plug a board into the PCI bus behind the 2nd host
bridge.  The board has registers in PCI I/O space.  An address is
assigned for those registers in the BAR in config space.  If you read
the BAR and then do an inb from that port number, you don't get the
I/O port on your board.

Ok then, that's what I described.

One solution that has been proposed is to set the base I/O port number
in the pci_dev structure to be actually the virtual address where you
can access that I/O port.  I don't like that solution because it means
that drivers for legacy PC-style devices can't do inb/outb to the
usual well-known port numbers and find the device they expect.  For
example, inb(0x3f8) won't access the first serial port (this is on
machines such as prep and some chrp which have a lot of PC-style
devices).

Ick...agree here.  Of course, the serial driver could be changed to
do other than inb/outb's for other archs and be passed the memory
mapped address of the ports.

My solution is to allocate say 1MB of I/O space per host bridge and
then adjust the pci_dev structures for the devices behind the 2nd and
subsequent host bridges.  So for example a board that has I/O ports at
0x1000 behind the 2nd host bridge would end up with its
pci_dev->resource[0].start == 0x101000.  The virtual <-> physical
mappings are set up so that the 2nd host bridge's I/O space is mapped
in starting at 0xff100000.  The result is that doing inb(0x101000)
accesses the device as expected.

This looks to be a good solution if we can't resolve to make the kernel
less legacy-oriented.  I changed the default K2 board memory map to
do a similar thing by orienting the 2nd host bridge right after the
1st in the physical map.  The virt->phys mapping was then 1:1 and things
work as you describe above.  Obviously, this approach won't work when
you have separate bridge ASICs since you can't get the memory map
granularity that I had in the dual host bridge package.

quoted

I recently did a port to the SBS K2 cPCI board which involves the
IBM CPC710 dual host bridge.  Due to the multi host bridge legacy
I/O difficulty, a documented assumption is that legacy I/O calls
would only be used on the "primary" host bridge.  I realize there
are plenty of drivers (like de4x5) that insist on using inw/outw
(and thus break on host bridge 2) but these drivers should be
fixed.

Fixed how?  I mean, how are you generically going to access PCI I/O
space without using inw/outw etc.?  On intel that is the only possible
way to do it so I just cannot see that you will persuade driver
authors that they shouldn't use inw/outw.

Well, on drivers like the de4x5 you switch to use the mem bar instead
of the I/O bar and then it's generic.  There are lots of drivers that
given the choice, chose to use I/O calls.  These can easily be fixed.

quoted

In the long run, the legacy I/O compatibility calls need to be
completely wiped out for non-x86 architectures.  It is all
memory mapped after all and only a few drivers like serial and
IDE would need separate low level access paths for x86/non-x86
architectures.

I disagree.  There will always be PCI devices with registers in PCI
I/O space.  I really don't want to see a situation where drivers get
littered with ifdefs because you have to use different access
functions for accessing PCI I/O space on non-intel machines and intel
machines.  That would be just totally counter-productive.

PCI I/O is memory mapped on everything but x86.  In most cases PCI
devices provide bars that map the same set of registers into both
I/O space and mem space.  That leaves only the true legacy devices
which have no memory bars to be concerned with.  It's a smaller
set of drivers that need different low level access methods than
you're making it out to be.

quoted

If we can't cover I/O with a BAT then it will definitely have some
ramifications with serial ports in the legacy I/O range among other
things.

Why?  Are you concerned about very early debugging?  If not, what?

Sure, that is a convenience of the BAT mapping.  It can all be
fixed though if that's the answer you're looking for. :)

--
Matt Porter
MontaVista Software, Inc.
mporter@mvista.com

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

On Tue, 19 Sep 2000, Matt Porter wrote:

On Wed, Sep 20, 2000 at 09:58:07AM +1100, Paul Mackerras wrote:

quoted

Matt Porter writes:

quoted

On Tue, Sep 19, 2000 at 02:59:02PM +1100, Paul Mackerras wrote:

One solution that has been proposed is to set the base I/O port number
in the pci_dev structure to be actually the virtual address where you
can access that I/O port.  I don't like that solution because it means
that drivers for legacy PC-style devices can't do inb/outb to the
usual well-known port numbers and find the device they expect.  For
example, inb(0x3f8) won't access the first serial port (this is on
machines such as prep and some chrp which have a lot of PC-style
devices).

Another option would be to use a translation table. If you limit I/O space to
64 kB (like on PC), you waste only 256 kB of memory (assuming a table with
32-bit pointers). With a multi-level tree or a smarter pointer scheme, you can
limit the waste even more. Since I/O accesses are intrinsically slow, the
overhead is minimal.

I agree this solution is not optimal, though.

Ick...agree here.  Of course, the serial driver could be changed to
do other than inb/outb's for other archs and be passed the memory
mapped address of the ports.

These days serial.c does support MMIO on PCI devices. In fact it supports
everything that looks even remotely like a NS16550 (hence it should be renamed
16550.c), both in I/O and in memory space.

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Matt Porter writes:

Ick...agree here.  Of course, the serial driver could be changed to
do other than inb/outb's for other archs and be passed the memory
mapped address of the ports.

And the parallel port driver, and the floppy driver, and the
keyboard/mouse drivers, and the ...

This looks to be a good solution if we can't resolve to make the kernel
less legacy-oriented.  I changed the default K2 board memory map to

If we only supported powermacs, and not preps or chrps, then we
probably could make the kernel less "legacy-oriented". :-) :-)

do a similar thing by orienting the 2nd host bridge right after the
1st in the physical map.  The virt->phys mapping was then 1:1 and things
work as you describe above.  Obviously, this approach won't work when
you have separate bridge ASICs since you can't get the memory map
granularity that I had in the dual host bridge package.

So we set up the virtual -> physical mapping to suit ourselves.  Not a
problem.

Well, on drivers like the de4x5 you switch to use the mem bar instead
of the I/O bar and then it's generic.  There are lots of drivers that
given the choice, chose to use I/O calls.  These can easily be fixed.

But what is wrong with using registers in I/O space and accessing them
with in*/out*?  I don't understand the aversion some people have shown
to using in*/out*.  They are the accessor functions for PCI I/O space,
that's all.  Whatever happens, we need accessor functions for PCI I/O
space.  We can call them something different or make them more
complicated but that would just create extra work for ourselves and
make it harder to port drivers between architectures.

PCI I/O is memory mapped on everything but x86.  In most cases PCI

So???  What we have is a level of abstraction that hides (from driver
authors) exactly how PCI memory and I/O space are accessed.  We have
readb/writeb etc. for PCI memory space and inb/outb etc. for PCI I/O
space.  The fact that on PPC both are mapped into the processor's
address space and accessed using loads and stores is something that
drivers should not care in the slightest about.  And in fact these
spaces are not accessed this way on some other non-intel platforms;
e.g. sparc64 uses special ldasi/stasi instructions.

devices provide bars that map the same set of registers into both
I/O space and mem space.  That leaves only the true legacy devices
which have no memory bars to be concerned with.  It's a smaller
set of drivers that need different low level access methods than
you're making it out to be.

The fact remains that a lot of drivers will continue to use inb/outb
etc., since they are developed primarily on the intel platform.  It
makes sense for us to have an inb/outb that works similarly to intel,
so that we can use as many drivers as possible with as little pain as
possible.

Paul.

--
Paul Mackerras, Senior Open Source Researcher, Linuxcare, Inc.
+61 2 6262 8990 tel, +61 2 6262 8991 fax
paulus@linuxcare.com.au, http://www.linuxcare.com.au/
Linuxcare.  Support for the revolution.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/