Re: xine, ppc and illegal instructions

Yup, I got the same illegal instructions in xine 0.4.01, and it would
sorta work running under gdb.  It seems to be dying in dlopen, which
is part of /lib/libdl.so.2.  On my system:

gwiz% rpm -q -f /lib/libdl.so.2
glibc-2.1.3-15f

A theory why xine 0.4.01 would work on Henry's system but not on mine
was that Henry was running with a newer glibc (2.2.something).  I was
going to try upgrading my glibc, but that turned out it was going to
require updating in excess of 60 RPMs to satisfy all the dependencies,
so I've temporarily shelved that idea.  YDL 2.0 is supposed to be
released RSN, which includes a 2.2 glibc, so I'll be upgrading to
that once it's available to see if it makes a difference (I may even
try using the 2.0 beta RPMs in advance of the official release).

If you want to try the stable 0.3.7 version of xine, I have a patch
to build it for PPC, and the 0.3.7 version doesn't encounter the
illegal instruction problem.  See my message from March 8 on the
xine-user e-mail archive for the patch (start at http://xine.sourceforge.net/).

There are two minor changes to that patch.

	*  In audio_out/audio_oss_out.c, the patch changed the constant
	   90000 to 88200.  It has since been pointed out to me that
	   this was incorrect and the constant should be left as 90000.

	*  In configure, there's a case check for "powerpc-*-linux*)".
	   This should be changed to "powerpc-*-linux* | ppc-*-linux*)".

Of course I really want to get to use 0.4.01 for its new functionality
such as audio/video synchronization and reported performance improvements,
but I'm stuck for the moment until I can upgrade to YDL 2.0.

						-Bill

On Sat, 24 Mar 2001, Henry Worth wrote:

quoted

Stefan Berndtsson [off-list ref] wrote:

quoted

I'll wait for the new patches and see if that changes >anything though.

Nothing in the patches for this problem, which is
quite recreatable, Bill Fink was looking into
this. If the esd daemon is running (just disabling desktop sounds is not enough, kill the
daemon) and
you run without -A esd, you will get ill. instr.
on ppc, unless running in gdb. ESD is useless
for this anyway, no hope of syncing A/V, at
least till they fix the esd_get_latency() call
which just hangs on both PPC and x86.

BTW - the xine user maillists would be a better
forum for this - <http://xine.sourceforge.net/>
I'm mailing you the cummalative patch.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

At 07:30 27.03.2001, Bill Fink wrote:

        *  In configure, there's a case check for "powerpc-*-linux*)".
           This should be changed to "powerpc-*-linux* | ppc-*-linux*)".

This isn't correct, it's always powerpc-*-linux*, the other one is
superflous. This usually happens if people didn't update their configure
scripts for a _long_ time and your patch is just papering over that.

Franz.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Some further investigation on my part and a question for Franz Sirl:

I updated to the latest Paulus rsync kernel (2.4.3-pre8) and the
latest gcc (2.95.3-2v) and 2.1 glibc (glibc-2.1.3-15g) from Franz,
but unfortunately none of this helps.  xine 0.4.01 still gets
illegal instructions (sometimes it gets a segmentation fault
instead).  No core dump is produced and it runs basically OK
when run under control of gdb, so it makes tracking down the
problem very difficult.

From debug printout, it appears that it is dying in a call to dlopen,

which is in libdl, which is part of glibc.  Here's the tail end of an
strace of a xine run:

open("/usr/local/install/xine-0.4.01/lib/xine/plugins", O_RDONLY|O_NONBLOCK|O_DI
RECTORY) = 7
fstat(7, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
fcntl(7, F_SETFD, FD_CLOEXEC)           = 0
lseek(7, 0, SEEK_CUR)                   = 0
getdents(7, /* 12 entries */, 2980)     = 284
open("/usr/local/install/xine-0.4.01/lib/xine/plugins/input_file.so", O_RDONLY)
= 8
fstat(8, {st_mode=S_IFREG|0755, st_size=17054, ...}) = 0
read(8, "\177ELF\1\2\1\0\0\0\0\0\0\0\0\0\0\3\0\24\0\0\0\1\0\0\t"..., 4096) = 409
6
mmap(0xfa28000, 70000, PROT_READ|PROT_EXEC, MAP_PRIVATE, 8, 0) = 0xfa28000
mprotect(0xfa29000, 65904, PROT_NONE)   = 0
mmap(0xfa38000, 8192, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 8,
0) = 0xfa38000
close(8)                                = 0
mprotect(0xfa28000, 4096, PROT_READ|PROT_WRITE) = 0
mprotect(0xfa28000, 4096, PROT_READ|PROT_EXEC) = 0

--- SIGILL (Illegal instruction) ---
+++ killed by SIGILL +++

Given that the last system call was an mprotect, I searched the archives
and discovered that Olaf Hering reported a problem with mprotect on
January 23 on the linuxppc-dev list, see:

http://lists.linuxppc.org/listarcs/linuxppc-dev/200101/msg00324.html

Olaf provided a patch to fix the problem, which I wasn't totally clear
exactly what it was against, but I'm assuming it was glibc (I've never
looked at the glibc source).  Assuming it was a glibc patch, it wouldn't
be in the glibc-2.1.3-15g I downloaded since that was dated January 14.

Now my question for Franz.  Do you think the problem/patch reported
by Olaf is likely related to the apparent dlopen/mprotect problem I'm
having with the 0.4.01 version of xine (note the 0.3.7 version works
fine)?  Or could it possibly be hardware related?  I'm running on a
G4 which has Altivec, while Henry's system is a G3 IIRC, which I believe
doesn't have Altivec (or some other hardware difference).

I've pulled down the source for glibc-2.1.3-15g, but before I actually
try rebuilding it on my system (with or without Olaf's patch), I'd
appreciate some expert advice about whether it's worth the effort or
likely to have any effect.

						-Thanks

						-Bill


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Bill Fink wrote:

I updated to the latest Paulus rsync kernel (2.4.3-pre8) and the
latest gcc (2.95.3-2v) and 2.1 glibc (glibc-2.1.3-15g) from Franz,
but unfortunately none of this helps.  xine 0.4.01 still gets
illegal instructions (sometimes it gets a segmentation fault
instead).

Have you tried other kernels? A lot of people including me have experienced
sporadic signals, most often with ldconfig, with Paulus' tree. I'm running
BenH's tree now and haven't seen it yet.

This is not to call Paulus' tree lame or anything. :) It's just that he
apparently has rather experimental stuff in it, which can always lead to
instabilities.


--
Earthling Michel Dänzer (MrCooper)    \   Debian GNU/Linux (powerpc) developer
CS student, Free Software enthusiast   \        XFree86 and DRI project member

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

On Sun, 1 Apr 2001, Michel [iso-8859-1] Dänzer wrote:

Bill Fink wrote:

quoted

I updated to the latest Paulus rsync kernel (2.4.3-pre8) and the
latest gcc (2.95.3-2v) and 2.1 glibc (glibc-2.1.3-15g) from Franz,
but unfortunately none of this helps.  xine 0.4.01 still gets
illegal instructions (sometimes it gets a segmentation fault
instead).

Have you tried other kernels? A lot of people including me have experienced
sporadic signals, most often with ldconfig, with Paulus' tree. I'm running
BenH's tree now and haven't seen it yet.

This is not to call Paulus' tree lame or anything. :) It's just that he
apparently has rather experimental stuff in it, which can always lead to
instabilities.

Hey, Grrrrrrrrrrrrrreattt!!!!!

That did the trick.  I rsync'ed BenH's latest 2.4.3 final kernel,
and now xine 0.4.01 works fine.  Now I just have to get MOL working
again with the latest kernels (probably just requires a rebuild).

Many, many thanks for the suggestion!

						-Bill


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Hi,

couple of  things:

- was the shared library loaded by dlopen properly built with -fPIC or -fpic?

- could there be a cache flushing bug?  When run in gdb if you set some
breakpoints in the shared library, it will flush the icache (to make sure the
breakpoint gets flushed out of data cache and any preloaded instruction are
thrown away) this in turn seems to "fix" the cache flush problem if one
exists.

- can you set ulimit -c unlimited and generate a core file from xine
  and post the resulting backtrace.

  If it looks empty, try the following:
  x/20i $lr -32
  in gdb just in case the problem was a branch out into the weeds

(if we are lucky lr will have the return address so that we can
possibly see at least where the bad branch comes from).  One way to get
an illegal instruction is to follow a calculated branch into the weeds
(happens sometimes in some C++ code during tailcalls when compiled above -O0).

Kevin

On Sunday 01 April 2001 13:51, Bill Fink wrote:

Some further investigation on my part and a question for Franz Sirl:

I updated to the latest Paulus rsync kernel (2.4.3-pre8) and the
latest gcc (2.95.3-2v) and 2.1 glibc (glibc-2.1.3-15g) from Franz,
but unfortunately none of this helps.  xine 0.4.01 still gets
illegal instructions (sometimes it gets a segmentation fault
instead).  No core dump is produced and it runs basically OK
when run under control of gdb, so it makes tracking down the
problem very difficult.

From debug printout, it appears that it is dying in a call to dlopen,
which is in libdl, which is part of glibc.  Here's the tail end of an
strace of a xine run:

open("/usr/local/install/xine-0.4.01/lib/xine/plugins",

O_RDONLY|O_NONBLOCK|O_DI

RECTORY) = 7
fstat(7, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
fcntl(7, F_SETFD, FD_CLOEXEC)           = 0
lseek(7, 0, SEEK_CUR)                   = 0
getdents(7, /* 12 entries */, 2980)     = 284
open("/usr/local/install/xine-0.4.01/lib/xine/plugins/input_file.so",

O_RDONLY)

= 8
fstat(8, {st_mode=S_IFREG|0755, st_size=17054, ...}) = 0
read(8, "\177ELF\1\2\1\0\0\0\0\0\0\0\0\0\0\3\0\24\0\0\0\1\0\0\t"..., 4096)

= 409

6
mmap(0xfa28000, 70000, PROT_READ|PROT_EXEC, MAP_PRIVATE, 8, 0) = 0xfa28000
mprotect(0xfa29000, 65904, PROT_NONE)   = 0
mmap(0xfa38000, 8192, PROT_READ|PROT_WRITE|PROT_EXEC,

MAP_PRIVATE|MAP_FIXED, 8,

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

- could there be a cache flushing bug?  When run in gdb if you set some
breakpoints in the shared library, it will flush the icache (to make sure the
breakpoint gets flushed out of data cache and any preloaded instruction are
thrown away) this in turn seems to "fix" the cache flush problem if one
exists.

Franz and I checked glibc for the cache flushing problem reported by
Samuel Rydth and it's ok. However, the kernel did miss those sync's.
I can't tell if it's the cause of the problem, however, my rsync kernel
have those changes. I'll push them to bk soon.

Ben.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

On Sun, 1 Apr 2001, Kevin B. Hendricks wrote:

Hi,

couple of  things:

- was the shared library loaded by dlopen properly built with -fPIC or -fpic?

I assume so.  It's Franz Sirl's latest 2.1 glibc RPM.

- could there be a cache flushing bug?  When run in gdb if you set some
breakpoints in the shared library, it will flush the icache (to make sure the
breakpoint gets flushed out of data cache and any preloaded instruction are
thrown away) this in turn seems to "fix" the cache flush problem if one
exists.

That makes sense to me.

- can you set ulimit -c unlimited and generate a core file from xine
  and post the resulting backtrace.

  If it looks empty, try the following:
  x/20i $lr -32
  in gdb just in case the problem was a branch out into the weeds

I was never able to get a core dump.  My coredumpsize is unlimited.
Earlier in the xine code, I could force a core dump by doing a "*0 = 1"
before the first call to pthread_create, but I couldn't get a core dump
by doing the same thing right after that call.

(if we are lucky lr will have the return address so that we can
possibly see at least where the bad branch comes from).  One way to get
an illegal instruction is to follow a calculated branch into the weeds
(happens sometimes in some C++ code during tailcalls when compiled above -O0).

As reported in my previous message, it's now working using the latest
BenH kernel (2.4.3 final) instead of the Paulus kernel (2.4.3-pre8) I
was previously trying with, so I'm happy.

						-Thanks

						-Bill


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Hi,

quoted

- could there be a cache flushing bug?  When run in gdb if you set some
breakpoints in the shared library, it will flush the icache (to make sure

the

quoted

breakpoint gets flushed out of data cache and any preloaded instruction

are

quoted

thrown away) this in turn seems to "fix" the cache flush problem if one
exists.

That makes sense to me.

As reported in my previous message, it's now working using the latest
BenH kernel (2.4.3 final) instead of the Paulus kernel (2.4.3-pre8) I
was previously trying with, so I'm happy.

Isn't Ben's rsync kernel the one with a change in the kernel cache flushing
code?

Samuel's message said the sequence should be

         stw     r4,0(r2)                        // store instruction
         dcbst   0,r2                            // Flush cache
         sync
         icbi    0,r2
         sync
         isync

I have sometimes seen code where you can batch things up with multiple  dcbf
and icbi before the final sync isync.

So does the batch version work properly on a 7400 series or not?

Either way, we should probably check the cache flushing code in XFree86  (it
has its own dynamic loader) and OpenOffice (bridges code uses self-modifying
code) and  the ppc closures code in libffi used by gcj to flush trampolines
and whatever ever other userland code uses cache flushing because many may
have been based on the kernel version of that routine.

I will check the latter two (OpenOffice and the gcj trampoline flush) since I
have easy access to that code.  Has anyone checked the XFree86 cache flushing
code?

Any info on if the flush a cache range with batched dcbf and icbi's version
used by the kernel works properly under 7400 would be greatly appreciated.

Kevin


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/