Re: Fwd: Re: still no accelerated X ($#!$*)

	The "b" constraint should be associated with "base_addr", not with
"regindex":

	asm("stwbrx %0,%1,%2": : "r"(regdata), "r"(regindex), "b"(base_addr));

David

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

On Thu, Jan 20, 2000, David Edelsohn [off-list ref] wrote:

The "b" constraint should be associated with "base_addr", not with
"regindex":

asm("stwbrx %0,%1,%2": : "r"(regdata), "r"(regindex), "b"(base_addr));

Hum... I still have to check what gcc/ppc specific constraints are. But
in this specific case, it's the index who should not be assigned to r0.
Both base and regdata can be r0. So either I'm mising something, or the
"b" constraint is actually wrong semanticall, or we need yet-another
constraint for the index.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

At 19:26 20.01.00 , David Edelsohn wrote:

        The "b" constraint should be associated with "base_addr", not with
"regindex":

        asm("stwbrx %0,%1,%2": : "r"(regdata), "r"(regindex),
"b"(base_addr));

Uhm, David, that's wrong. "b" has to be assigned to %1!

Franz.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

On Thu, 20 Jan 2000, Franz Sirl wrote:

At 19:26 20.01.00 , David Edelsohn wrote:

quoted

        The "b" constraint should be associated with "base_addr", not with
"regindex":

        asm("stwbrx %0,%1,%2": : "r"(regdata), "r"(regindex),
"b"(base_addr));

Uhm, David, that's wrong. "b" has to be assigned to %1!

Actually if base_addr can be reused by the compiler for other accesses
to the same area (byte or big endian), it should be written as:

"stwbrx %0,%1,%2": : "r" (regdata), "b" (base_addr), "r" (regindex)

with a volatile qualifier on the asm statement but I disagree on the
"memory" clobber if this does not access areas the compiler will ever
touch and does not have side effect.

There are already too many memory clobbers out there, they are bad because
they basically tell the compiler that it can not keep a single variable
in a register. Have a look at the effect of in and out instructions
in the linux kernel which reload isa_io_base repeatedly when there is no
need for it, that's not exactly a speed issue but a icache footprint
and bloat issue.

This is an effect which is important on all machines which have a
resonable number of registers (all modern ones in practice and perhaps
even the 68k).

	Gabriel.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Hi,

Actually if base_addr can be reused by the compiler for other accesses
to the same area (byte or big endian), it should be written as:

"stwbrx %0,%1,%2": : "r" (regdata), "b" (base_addr), "r" (regindex)

with a volatile qualifier on the asm statement but I disagree on the
"memory" clobber if this does not access areas the compiler will ever
touch and does not have side effect.

There are already too many memory clobbers out there, they are bad because
they basically tell the compiler that it can not keep a single variable
in a register.

In this particular case, the base address can change (but very very rarely
such as writing to one Aperature or Another on the Rage 128 card) and all of
the writes are made to either the card memory mapped io or the frame buffer
itself.

Should I not include the : "memory" clobber in this case?  Will it hurt
performance much?

Thanks,

Kevin

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

On Thu, 20 Jan 2000, Kevin Hendricks wrote:

quoted

Actually if base_addr can be reused by the compiler for other accesses
to the same area (byte or big endian), it should be written as:

"stwbrx %0,%1,%2": : "r" (regdata), "b" (base_addr), "r" (regindex)

with a volatile qualifier on the asm statement but I disagree on the
"memory" clobber if this does not access areas the compiler will ever
touch and does not have side effect.

There are already too many memory clobbers out there, they are bad because
they basically tell the compiler that it can not keep a single variable
in a register.

In this particular case, the base address can change (but very very rarely
such as writing to one Aperature or Another on the Rage 128 card) and all of
the writes are made to either the card memory mapped io or the frame buffer
itself.

The base_addr may change in between calls, but not while executing the asm
code. The memory clobber means that memory contents may change while
executing the asm code without gcc knowing about it.

Gr{oetje,eeting}s,
--
Geert Uytterhoeven -- Linux/{m68k~Amiga,PPC~CHRP} -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

	Hi,

In this particular case, the base address can change (but very very rarely
such as writing to one Aperature or Another on the Rage 128 card) and all of
the writes are made to either the card memory mapped io or the frame buffer
itself.

Then the memory clobber would force the compiler to reload base_addr
between 2 writes to the frame buffer.

Should I not include the : "memory" clobber in this case?  Will it hurt
performance much?

I think that it is not necessary: the best thing with a compiler which
performs alias analysis might be to tell the truth

asm ("stwbrx %1,%2,%3"
     : "=m" (*(volatile unsigned *)(base_addr+regindex))
     : "r" (regdata), "b" (base_addr), "r" (regindex));

Note we don't use %0, and it won't produce any aditional code. You may
want to check what the compiler would have generated as addressing mode
by appending " # %0" at the end of the code string.

I truly wish there were a constraint for indexed addressing modes (anyway
this will be necessary for proper Altivec support).

	Gabriel.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Am Don, 20 Jan 2000 schrieb Gabriel Paubert:

Hi,

quoted

In this particular case, the base address can change (but very very rarely
such as writing to one Aperature or Another on the Rage 128 card) and all of
the writes are made to either the card memory mapped io or the frame buffer
itself.

Then the memory clobber would force the compiler to reload base_addr
between 2 writes to the frame buffer.

quoted

Should I not include the : "memory" clobber in this case?  Will it hurt
performance much?

I think that it is not necessary: the best thing with a compiler which
performs alias analysis might be to tell the truth

asm ("stwbrx %1,%2,%3"
    : "=m" (*(volatile unsigned *)(base_addr+regindex))
    : "r" (regdata), "b" (base_addr), "r" (regindex));

Note we don't use %0, and it won't produce any aditional code. You may
want to check what the compiler would have generated as addressing mode
by appending " # %0" at the end of the code string.

It depends a little bit on the usage of the asm's if the memory (either
global or local) clobber is needed or not. If you use them for read/writes to HW
registers needing ordering (which is very likely here since we talk about
graphics HW), the compiler can only decide on the memory usage defined by the
clobbers/memory inputs on how to order the inlines (volatile has no effect on
this).

Actually the load instructions need a memory input too:

asm volatile ("lwbrx %0,%1,%2" : "=r"(val) : "b"(regindex), "r"(base_addr),
"m" (*(volatile unsigned *)(base_addr+regindex)));

And to insure ordering on processor level you still need the eieio (with a
memory clobber) as usual.

Franz.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

On Thu, 20 Jan 2000, Franz Sirl wrote:

It depends a little bit on the usage of the asm's if the memory (either
global or local) clobber is needed or not. If you use them for read/writes to HW
registers needing ordering (which is very likely here since we talk about
graphics HW), the compiler can only decide on the memory usage defined by the
clobbers/memory inputs on how to order the inlines (volatile has no effect on
this).

Do you mean that a volatile memory reference can be reordered wit an asm
volatile statement ? I thought theat this was not possible.

From GCC's documentation:

   "You can prevent an `asm' instruction from being deleted, moved
significantly, or combined, by writing the keyword `volatile' after the
`asm'."

You might disagree, but I consider that moving across a volatile memory
reference is a _significant_ move, a very significant one.

Actually the load instructions need a memory input too:

asm volatile ("lwbrx %0,%1,%2" : "=r"(val) : "b"(regindex), "r"(base_addr),
"m" (*(volatile unsigned *)(base_addr+regindex)));

Indeed, perhaps even more than the store since asm without output operands
are actually assumed to have side effects.

And to insure ordering on processor level you still need the eieio (with a
memory clobber) as usual.

The best thing to do is probably to put the eieio inside the asm
statement, perhaps with 2 macros, one that includes the eieio and one that
does not include it.

Memory clobbers should be used extremely sparingly, I view them as the
last resort when _nothing_ else would work. They are used far too often
for my taste in the kernel, probably because they don't affect
significantly machines with very few registers like Intel which have to
permanently spill registers to memory or stack.

	Gabriel.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Hi,

I don't want to be dense here but why two macros (one with eieio and one without
eieio).  Can two processors in an SMP setting be trying to drive the same
video card at the same time?   I didn't think that was possible.  I saw the
eieio usage in the kernel versions and in aty128fb.c but thought that multiple
processors might use those macros at the same time and multiple processors
might have different fbdev drivers running in multi-head applications.  But I
thought only one processor could drive a video hardware card.  Is this a bad
assumption?

So exactly what is the best way to write these macros for Xpmac?  Using
the output constraints approach with eieio following it or is all of this
overkill.

From the various posts (given the operand ordering done in the original post),

here is what I have tried to piece together.

asm volatile ("stwbrx %1,%2,%3; eieio" : "=m" (*(volatile unsigned
*)(base_addr+regindex))       : "r" (regdata), "b" (regindex), "r" (base_addr));

asm volatile ("lwbrx %0,%1,%2; eieio" : "=r"(val) : "b"(regindex),
"r"(base_addr), "m" (*(volatile unsigned *)(base_addr+regindex)));


Please let me know how to change the above so that I get it right this time.

Thanks,

Kevin


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

	Hi,

I don't want to be dense here but why two macros (one with eieio and one without
eieio).  Can two processors in an SMP setting be trying to drive the same
video card at the same time?   I didn't think that was possible.  I saw the
eieio usage in the kernel versions and in aty128fb.c but thought that multiple
processors might use those macros at the same time and multiple processors
might have different fbdev drivers running in multi-head applications.  But I
thought only one processor could drive a video hardware card.  Is this a bad
assumption?

Multiple processors attempting to access the video hardware should be
serialized one way or anoter (semaphores or whatever).

So exactly what is the best way to write these macros for Xpmac?  Using
the output constraints approach with eieio following it or is all of this
overkill.

eieio may be useful and even necessary when accessing the registers, but
in the frame buffer section it prevents the processor from performing
store merging to non guarded memory and other optimizations (like the
bridge to merge the writes and transforming them into bursts to the
framebuffer).

eieio is also necessary to avoid load from being moved before stores when
accessing IO registers: if your intent is to write to register A and then
read register B and that the first write actually changes the contents of
register B (for example your trigger an operation with the write and wait
for it to finish by reading register B which contains a busy bit), then
the eieio is necessary (it is not necessary if the registers are the
same).

quoted

From the various posts (given the operand ordering done in the original post),

here is what I have tried to piece together.

asm volatile ("stwbrx %1,%2,%3; eieio" : "=m" (*(volatile unsigned
*)(base_addr+regindex))       : "r" (regdata), "b" (regindex), "r" (base_addr));

asm volatile ("lwbrx %0,%1,%2; eieio" : "=r"(val) : "b"(regindex),
"r"(base_addr), "m" (*(volatile unsigned *)(base_addr+regindex)));

I actually doubt that the eieio are necessary but then I'm not a
specialist on this kind of hardware. Every eieio is a bus broadcast
operation (except on 603, on G3 it is IIRC an option controlled by a bit
in HID0) and actually has a cost comparable to a write posted I/O access
but the other consequences (preventing bursts on the I/O bus) may actually
cause a significant performance hit.  So it should be used only when
necessary...

Please let me know how to change the above so that I get it right this time.

Try to determine first whether the eieio are necessary; for access to the
frame buffer I'm almost sure that they are superfluous and potentially
very costly in terms of performance. For the MMIO I suspect that they may
be necessary at some places, but adding them systematically will have less
impact.

	Gabriel.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Hi,

I actually doubt that the eieio are necessary but then I'm not a
specialist on this kind of hardware. Every eieio is a bus broadcast
operation (except on 603, on G3 it is IIRC an option controlled by a bit
in HID0) and actually has a cost comparable to a write posted I/O access
but the other consequences (preventing bursts on the I/O bus) may actually
cause a significant performance hit.  So it should be used only when
necessary...

quoted

Please let me know how to change the above so that I get it right this time.

Try to determine first whether the eieio are necessary; for access to the
frame buffer I'm almost sure that they are superfluous and potentially
very costly in terms of performance. For the MMIO I suspect that they may
be necessary at some places, but adding them systematically will have less
impact.

Okay, I went and looked at the latest aty128fb.c code and it does not use eieio
anywhere.  I looked at ealier verions of this file and it at one time had eieio
but they have since been removed.

I also looked and the endian conversion routines do not use the output
contraint approach you took but do include the memory clobber on the writes.

I think I will go with the output constraint version given above without the
eieio until or unless the kernel driver begins to use them too.

Thanks for all of your help with this everyone.  I have learned alot.

Kevin

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

On Thu, 20 Jan 2000, Kevin Hendricks wrote:

quoted

I actually doubt that the eieio are necessary but then I'm not a
specialist on this kind of hardware. Every eieio is a bus broadcast
operation (except on 603, on G3 it is IIRC an option controlled by a bit
in HID0) and actually has a cost comparable to a write posted I/O access
but the other consequences (preventing bursts on the I/O bus) may actually
cause a significant performance hit.  So it should be used only when
necessary...

quoted

Please let me know how to change the above so that I get it right this time.

Try to determine first whether the eieio are necessary; for access to the
frame buffer I'm almost sure that they are superfluous and potentially
very costly in terms of performance. For the MMIO I suspect that they may
be necessary at some places, but adding them systematically will have less
impact.

Okay, I went and looked at the latest aty128fb.c code and it does not use eieio
anywhere.  I looked at ealier verions of this file and it at one time had eieio
but they have since been removed.

Perhaps they were replaced by the platform independent wmb()?

Gr{oetje,eeting}s,
--
Geert Uytterhoeven -- Linux/{m68k~Amiga,PPC~CHRP} -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

On Thu, Jan 20, 2000, Kevin Hendricks [off-list ref] wrote:

From the various posts (given the operand ordering done in the original

post),

here is what I have tried to piece together.

asm volatile ("stwbrx %1,%2,%3; eieio" : "=m" (*(volatile unsigned
*)(base_addr+regindex))       : "r" (regdata), "b" (regindex), "r"
(base_addr));

asm volatile ("lwbrx %0,%1,%2; eieio" : "=r"(val) : "b"(regindex),
"r"(base_addr), "m" (*(volatile unsigned *)(base_addr+regindex)));

Hi Kevin !

A good rule is to use eieio() when accessing a register (that means doing
an access that actually performs an action and whose ordering is
important relative to other accesses of the same type) and not use it
when filling the framebuffer. There are usually few enough register
accesses for this to work. it may be optimal to skip eieio's when writing
to a bunch "parameters" registers where ordering is not important, in
this case you just need to put an eieio() between those, and the register
write that triggers the engine operation that will use those parameters.
However, when doing that, the PCI bridge is allowed to combine your
register writes in a burst, and I know some cards who don't handle burst
access to MMIO registers very well.

Basically, eieio() will make sure that all previous memory accesses will
have been finished before memory accesses after the eieio are done.

It may be important to make sure that the last bit of framebuffer has
been written before "starting" an engine operation. So one eieio between
frame buffer filling and engine register access may be useful in the case
where you use eieio after the write in the asm.

I personally tend to prefer doing the eieio _before_ the read/write in
the asm code, but there are some rare cases where you mix eieio and
non-eieio accesses (like with your framebuffer) where special care must
be taken and may require both eieio before and after the register access.

Another thing to take care of is PCI write posting: Basically, when you
write, let's say, a MMIO register, you are not guaranteed that this write
have actually been done unless you do a read from the same io space. For
example: If you write an interrupt mask register to disable an interrupt
followed by critical code in which this interrupt _must not_ happen, you
need absolutely to do a read (typically to re-read the mask you just
wrote to) after the write, and before the critical code.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

On Fri, 21 Jan 2000, Benjamin Herrenschmidt wrote:

Hi Kevin !

A good rule is to use eieio() when accessing a register (that means doing
an access that actually performs an action and whose ordering is
important relative to other accesses of the same type) and not use it
when filling the framebuffer. There are usually few enough register
accesses for this to work. it may be optimal to skip eieio's when writing
to a bunch "parameters" registers where ordering is not important, in
this case you just need to put an eieio() between those, and the register
write that triggers the engine operation that will use those parameters.

Indeed, that's what I do in my VME driver. I did not like the fact that
readl/writel always insert an eieio. This is safe but almost doubles the
number of bus cycles. Very often it translates into:

- a bunch or register accesses,
- eieio,
- the access which triggers the operation (which was a read in some cases
  for the VME driver).
- eieio again, so you don't have problems with setting up the next
operation...

However, when doing that, the PCI bridge is allowed to combine your
register writes in a burst, and I know some cards who don't handle burst
access to MMIO registers very well.

Broken HW exists but in a device specific driver, you should know whether
it is necessary or not from the errata...

Basically, eieio() will make sure that all previous memory accesses will
have been finished before memory accesses after the eieio are done.

It may be important to make sure that the last bit of framebuffer has
been written before "starting" an engine operation. So one eieio between
frame buffer filling and engine register access may be useful in the case
where you use eieio after the write in the asm.

If there is one before the register access that triggers the engine
operation, it should be enough.

I personally tend to prefer doing the eieio _before_ the read/write in
the asm code, but there are some rare cases where you mix eieio and
non-eieio accesses (like with your framebuffer) where special care must
be taken and may require both eieio before and after the register access.

That's a good quetion, does anybody have any final conclusion about
whether it is better to ensure ordering before or after the access ?

Another thing to take care of is PCI write posting: Basically, when you
write, let's say, a MMIO register, you are not guaranteed that this write
have actually been done unless you do a read from the same io space. For
example: If you write an interrupt mask register to disable an interrupt
followed by critical code in which this interrupt _must not_ happen, you
need absolutely to do a read (typically to re-read the mask you just
wrote to) after the write, and before the critical code.

In this case, I think that you need the following:

- write,
- eieio,
- read,
- isync to make sure that the read has reached the registers and is not in
a load pending queue or whatever which can be quite deep especially if the
processor does never need the result of the read...

	Gabriel.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

On Fri, Jan 21, 2000, Gabriel Paubert [off-list ref] wrote:

- write,
- eieio,
- read,
- isync to make sure that the read has reached the registers and is not in
a load pending queue or whatever which can be quite deep especially if the
processor does never need the result of the read...

Indeed. Some time ago, I fixed the pmac-pic mask/unmask routines this way:

 - setup new mask in cached_mask (variable)
 - write_mask(cached_mask)
 - do {
 -   sync();
 - } while (read_mask() != cached_mask)

Note that both read_mask and write_mask will do eieio.

I beleive the sync could be replaced by an isync, I'm just not 100% sure
of the SMP behaviour but the mask/unmask routines should be fully
synchronized anyway. (And they are called with EE off).

Without this fix, we occasionally had bogus interrupts coming from the
IDE and possibly other rare problems.

I added a smiliar fix to the openpic code in my recent kernels too.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

On Thu, Jan 20, 2000, Kevin Hendricks [off-list ref] wrote:

Okay, I went and looked at the latest aty128fb.c code and it does not use
eieio
anywhere.  I looked at ealier verions of this file and it at one time had
eieio
but they have since been removed.

I also looked and the endian conversion routines do not use the output
contraint approach you took but do include the memory clobber on the writes.

I just looked at atyfb.c and aty128fb.c in my source tree (atyfb is
2.2.14 one and aty128fb is the latest backport done by atong) and neither
uses eieio nor mb(), wmb(), ...

This looks bogus to me. I've spotted a few cases where those calls should
be in.

We can either put the eieio back in the access functions (less optimal,
but we can also fix the constraints to get rid of the memory clobber as
discussed previously), or we can fill the code with carefuly placed mb()
and wmb() but this requires more knowledge of the chipset than I actually
have.

I'll put back eieio() in the access macros for my kernels until a
definitive answer pops up on this issue.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

At 01:05 21.01.00 , Gabriel Paubert wrote:

On Thu, 20 Jan 2000, Franz Sirl wrote:

quoted

It depends a little bit on the usage of the asm's if the memory (either
global or local) clobber is needed or not. If you use them for

read/writes to HW

quoted

registers needing ordering (which is very likely here since we talk about
graphics HW), the compiler can only decide on the memory usage defined

by the

quoted

clobbers/memory inputs on how to order the inlines (volatile has no

effect on

quoted

this).

Do you mean that a volatile memory reference can be reordered wit an asm
volatile statement ? I thought theat this was not possible.

quoted

From GCC's documentation:

   "You can prevent an `asm' instruction from being deleted, moved
significantly, or combined, by writing the keyword `volatile' after the
`asm'."

You might disagree, but I consider that moving across a volatile memory
reference is a _significant_ move, a very significant one.

asm volatile doesn't declare any volatile memory references! What makes you
think it does?
There was a lengthy discussion about this with Richard Henderson on the
gcc/egcs lists a while ago. The result was a rewrite of
linux/include/asm-ppc/io.h in the kernel to it's current state.

Franz.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

On Fri, 21 Jan 2000, Franz Sirl wrote:

quoted

   "You can prevent an `asm' instruction from being deleted, moved
significantly, or combined, by writing the keyword `volatile' after the
`asm'."

You might disagree, but I consider that moving across a volatile memory
reference is a _significant_ move, a very significant one.

asm volatile doesn't declare any volatile memory references! What makes you
think it does?

No it does not declare any volatile memory reference, but it is
"guaranteed not to be moved significantly". What does this exactly mean ?

Anyway asm volatile statements are guranteed not to reordered wrt each
other. So let us illustrate it with examples; if you write:

asm volatile("stwbrx...", ...);

asm volatile("eieio");

asm volatile("lwbrx...", ...);

the compiler should respect the order even without nasty memory clobbers.
The funny thing is that if the accesses are big endian or byte, then you
don't need the asm and a reordering might perhaps take place. In this
case:

*(volatile unsigned *)output_reg = data;

asm volatile("eieio");

result = *(volatile unsigned *) input_reg;

then the eieio might move become the first or last statement according to
what you say. I consider that this does not reflect the documentation
since moving across a volatile memory reference is a significant move
in my book.

Note that the documentation also states clearly that such an asm without
any parameters is considered as having undeterminate side effects so it
might actually not be moved, but it might be as bad as a memory clobber.

There was a lengthy discussion about this with Richard Henderson on the
gcc/egcs lists a while ago. The result was a rewrite of
linux/include/asm-ppc/io.h in the kernel to it's current state.

I remember, but my goal is a memory-clobber-free kernel, although there
are many other efficiency issues right now in Linux/PPC...

	Gabriel
	(desperately trying to build gcc_latest_snapshot)


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/