From: Bill Gatliff <hidden> Date: 2007-07-16 16:02:13
Guys:
I'm trying to track down a problem where gdbserver is issuing a bogus
syscall at the onset of a debugging session. I'm using gdb-6.6, on an
(ancient, I know) linux-2.4.16 kernel, pc603e machine. Gcc-3.4.5,
glibc-2.2.5 (both built with crosstools-0.43). Gdbserver is statically
linked.
When I run gdbserver under strace on the target, I see these in the log
shortly after initiating the connection from my workstation:
...
ptrace(PTRACE_PEEKTEXT, 947, 0x10405394, [0x103e0cb0]) = 0
ptrace(PTRACE_PEEKTEXT, 947, 0x10405398, [0x103e0ce4]) = 0
ptrace(PTRACE_PEEKTEXT, 947, 0x1040539c, [0x103e0ce8]) = 0
send(4, "$103df2cc103df2e8103df2ec103df2f"..., 644, 0) = 644
recv(4, 0x7ffffd60, 1, 0) = ? ERESTARTSYS (To be restarted)
--- SIGIO (I/O possible) @ 0 (0) ---
syscall_4294966784(0xa, 0x7ffffd34, 0x1, 0, 0x1008a3c7, 0x1008b5a3,
0x1008b5a4, 0, 0x1, 0x80808080, 0x1008e778, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0x10080000, 0x10080000, 0x10080000, 0x1008b5a3, 0x10080000,
0x1008b320, 0x100291a4, 0xd032, 0xa) = -1 (errno 38)
write(2, "putpkt(read): Function not imple"..., 39) = 39
...
Note the bogus syscall argument 4294966784. It isn't *completely*
bogus, interestingly, since if you google for that you come up with a
few hits. But no resolutions.
I'm stumped. Does this problem sound familiar to anyone? Had the same
problem with gcc-2.95.3.
Kindest regards,
b.g.
--
Bill Gatliff
bgat@billgatliff.com
That's -512, a.k.a. the errno value used by syscall restarting. I'd
say your glibc does not obey the restartable syscall convention used
by your kernel, and when it tries to restart the syscall the errno
value is not being replaced by the syscall number. Check the assembly
for recv.
--
Daniel Jacobowitz
CodeSourcery
That's -512, a.k.a. the errno value used by syscall restarting. I'd
say your glibc does not obey the restartable syscall convention used
by your kernel, and when it tries to restart the syscall the errno
value is not being replaced by the syscall number. Check the assembly
for recv.
Very good catch! Thanks soooo much. Here's the code, from my libc.a:
00000000 <__libc_recv>:
0: 94 21 ff d0 stwu r1,-48(r1)
4: 90 61 00 14 stw r3,20(r1)
8: 90 81 00 18 stw r4,24(r1)
c: 90 a1 00 1c stw r5,28(r1)
10: 90 c1 00 20 stw r6,32(r1)
14: 81 42 00 0c lwz r10,12(r2)
18: 2c 0a 00 00 cmpwi r10,0
1c: 40 82 00 20 bne- 3c <__libc_recv+0x3c>
20: 38 60 00 0a li r3,10
24: 38 81 00 14 addi r4,r1,20
28: 38 00 00 66 li r0,102
2c: 44 00 00 02 sc
30: 38 21 00 30 addi r1,r1,48
34: 4c a3 00 20 bnslr+
38: 48 00 00 00 b 38 <__libc_recv+0x38>
Again, this is 603e on linux-2.4.16 glibc-2.2.5 gcc-2.95.3. (Odd, I
can't seem to find this function in a statically-linked gdbserver, nor
any reference to it in the gdbserver-6.5 source code).
On the kernel side:
_GLOBAL(DoSyscall)
...
blrl /* Call handler */
.globl ret_from_syscall_1
ret_from_syscall_1:
20: stw r3,RESULT(r1) /* Save result */
li r10,-_LAST_ERRNO
cmpl 0,r3,r10
blt 30f
neg r3,r3
cmpi 0,r3,ERESTARTNOHAND
bne 22f
li r3,EINTR
22: lwz r10,_CCR(r1) /* Set SO bit in CR */
oris r10,r10,0x1000
stw r10,_CCR(r1)
30: stw r3,GPR3(r1) /* Update return value */
b ret_from_except
...
ret_from_except:
...
lwz r3,_CCR(r1)
...
mtcrf 0xFF,r3
...
RFI
Now, I'm a little rusty on PPC asm (I've been doing a lot of ARM
lately), but it looks to me like the kernel is setting bit 0 in CR0
(oris r10, r10, 0x1000) a.k.a LT, but the user side is looking at CR0
(bnslr+) bit 3 a.k.a. SO. Or maybe the other way around, I'm not sure
after reading Sections 1.2 and 2.1 of the Programming Environments manual.
Or am I misinterpreting something? I must be, this is well-trodden code
I'm thinking...
The readchar() in gdbserver's remote-utils.c just calls read() on the
file descriptor for the socket. Still trying to track that code down...
b.g.
--
Bill Gatliff
bgat@billgatliff.com
From: Daniel Jacobowitz <hidden> Date: 2007-07-18 19:00:23
On Wed, Jul 18, 2007 at 12:59:42PM -0500, Bill Gatliff wrote:
Now, I'm a little rusty on PPC asm (I've been doing a lot of ARM
lately), but it looks to me like the kernel is setting bit 0 in CR0
(oris r10, r10, 0x1000) a.k.a LT, but the user side is looking at CR0
(bnslr+) bit 3 a.k.a. SO. Or maybe the other way around, I'm not sure
after reading Sections 1.2 and 2.1 of the Programming Environments manual.
It's not checking for restart here - userspace isn't supposed to have to.
It's probably checking for error. Check for the bit of kernel code
that's supposed to back you up two instructions.
--
Daniel Jacobowitz
CodeSourcery
From: Bill Gatliff <hidden> Date: 2007-07-23 15:37:32
Daniel Jacobowitz wrote:
On Wed, Jul 18, 2007 at 12:59:42PM -0500, Bill Gatliff wrote:
quoted
Now, I'm a little rusty on PPC asm (I've been doing a lot of ARM
lately), but it looks to me like the kernel is setting bit 0 in CR0
(oris r10, r10, 0x1000) a.k.a LT, but the user side is looking at CR0
(bnslr+) bit 3 a.k.a. SO. Or maybe the other way around, I'm not sure
after reading Sections 1.2 and 2.1 of the Programming Environments manual.
It's not checking for restart here - userspace isn't supposed to have to.
It's probably checking for error. Check for the bit of kernel code
that's supposed to back you up two instructions.
I don't see it in this kernel. What I see is this after the call to the
syscall handler:
li r10,-_LAST_ERRNO
cmpl 0,r3,r10
blt 30f
neg r3,r3
cmpi 0,r3,ERESTARTNOHAND
bne 22f
li r3,EINTR
22: lwz r10,_CCR(r1) /* Set SO bit in CR */
oris r10,r10,0x1000
stw r10,_CCR(r1)
30: stw r3,GPR3(r1) /* Update return value */
b ret_from_except
66: li r3,ENOSYS
b 22b
?
--
Bill Gatliff
bgat@billgatliff.com
From: Bill Gatliff <hidden> Date: 2007-07-23 15:38:41
Daniel Jacobowitz wrote:
On Wed, Jul 18, 2007 at 12:59:42PM -0500, Bill Gatliff wrote:
quoted
Now, I'm a little rusty on PPC asm (I've been doing a lot of ARM
lately), but it looks to me like the kernel is setting bit 0 in CR0
(oris r10, r10, 0x1000) a.k.a LT, but the user side is looking at CR0
(bnslr+) bit 3 a.k.a. SO. Or maybe the other way around, I'm not sure
after reading Sections 1.2 and 2.1 of the Programming Environments manual.
It's not checking for restart here - userspace isn't supposed to have to.
It's probably checking for error. Check for the bit of kernel code
that's supposed to back you up two instructions.
I don't see it in this kernel. What I see is this after the call to the
syscall handler:
li r10,-_LAST_ERRNO
cmpl 0,r3,r10
blt 30f
neg r3,r3
cmpi 0,r3,ERESTARTNOHAND
bne 22f
li r3,EINTR
22: lwz r10,_CCR(r1) /* Set SO bit in CR */
oris r10,r10,0x1000
stw r10,_CCR(r1)
30: stw r3,GPR3(r1) /* Update return value */
b ret_from_except
66: li r3,ENOSYS
b 22b
?
--
Bill Gatliff
bgat@billgatliff.com
From: Bill Gatliff <hidden> Date: 2007-07-23 16:06:23
Daniel Jacobowitz wrote:
On Wed, Jul 18, 2007 at 12:59:42PM -0500, Bill Gatliff wrote:
quoted
Now, I'm a little rusty on PPC asm (I've been doing a lot of ARM
lately), but it looks to me like the kernel is setting bit 0 in CR0
(oris r10, r10, 0x1000) a.k.a LT, but the user side is looking at CR0
(bnslr+) bit 3 a.k.a. SO. Or maybe the other way around, I'm not sure
after reading Sections 1.2 and 2.1 of the Programming Environments manual.
It's not checking for restart here - userspace isn't supposed to have to.
It's probably checking for error. Check for the bit of kernel code
that's supposed to back you up two instructions.
I don't see it in this kernel. What I see is this after the call to the
syscall handler:
li r10,-_LAST_ERRNO
cmpl 0,r3,r10
blt 30f
neg r3,r3
cmpi 0,r3,ERESTARTNOHAND
bne 22f
li r3,EINTR
22: lwz r10,_CCR(r1) /* Set SO bit in CR */
oris r10,r10,0x1000
stw r10,_CCR(r1)
30: stw r3,GPR3(r1) /* Update return value */
b ret_from_except
66: li r3,ENOSYS
b 22b
?
--
Bill Gatliff
bgat@billgatliff.com
From: Daniel Jacobowitz <hidden> Date: 2007-07-23 16:15:18
On Mon, Jul 23, 2007 at 11:06:13AM -0500, Bill Gatliff wrote:
Daniel Jacobowitz wrote:
quoted
On Wed, Jul 18, 2007 at 12:59:42PM -0500, Bill Gatliff wrote:
quoted
Now, I'm a little rusty on PPC asm (I've been doing a lot of ARM
lately), but it looks to me like the kernel is setting bit 0 in CR0
(oris r10, r10, 0x1000) a.k.a LT, but the user side is looking at CR0
(bnslr+) bit 3 a.k.a. SO. Or maybe the other way around, I'm not sure
after reading Sections 1.2 and 2.1 of the Programming Environments manual.
It's not checking for restart here - userspace isn't supposed to have to.
It's probably checking for error. Check for the bit of kernel code
that's supposed to back you up two instructions.
I don't see it in this kernel. What I see is this after the call to the
syscall handler:
Look around do_signal:
regs->nip -= 4; /* Back up & retry system call */
If your kernel has corrupted the register containing the syscall
number at this point, that would explain your problem. It will then
do the wrong syscall. I guess PPC only backs up one instruction.
--
Daniel Jacobowitz
CodeSourcery
From: Bill Gatliff <hidden> Date: 2007-07-26 16:29:04
Daniel Jacobowitz wrote:
Look around do_signal:
regs->nip -= 4; /* Back up & retry system call */
If your kernel has corrupted the register containing the syscall
number at this point, that would explain your problem. It will then
do the wrong syscall. I guess PPC only backs up one instruction.
Found the code in do_signal, right where you said it would be. I threw
in a printk or two to see what's up when an ERESTARTSYS is detected, and
started getting OOPSes all over the place. Apparently that's not a good
place for printk. :)
Before I go on, can anyone confirm that gdbserver and/or strace _ever_
worked with circa 2.4.16 PPC kernels?
b.g.
--
Bill Gatliff
bgat@billgatliff.com
From: Andreas Schwab <hidden> Date: 2007-07-23 16:19:43
Bill Gatliff [off-list ref] writes:
Daniel Jacobowitz wrote:
quoted
On Wed, Jul 18, 2007 at 12:59:42PM -0500, Bill Gatliff wrote:
quoted
Now, I'm a little rusty on PPC asm (I've been doing a lot of ARM
lately), but it looks to me like the kernel is setting bit 0 in CR0
(oris r10, r10, 0x1000) a.k.a LT, but the user side is looking at CR0
(bnslr+) bit 3 a.k.a. SO.
Bits are numbered from left to right, thus 0x10000000 is bit 3 of CR0
Andreas.
--
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5
"And now for something completely different."
That's -512, a.k.a. the errno value used by syscall restarting. I'd
say your glibc does not obey the restartable syscall convention used
by your kernel, and when it tries to restart the syscall the errno
value is not being replaced by the syscall number. Check the assembly
for recv.
Very good catch! Thanks soooo much. Here's the code, from my libc.a:
00000000 <__libc_recv>:
0: 94 21 ff d0 stwu r1,-48(r1)
4: 90 61 00 14 stw r3,20(r1)
8: 90 81 00 18 stw r4,24(r1)
c: 90 a1 00 1c stw r5,28(r1)
10: 90 c1 00 20 stw r6,32(r1)
14: 81 42 00 0c lwz r10,12(r2)
18: 2c 0a 00 00 cmpwi r10,0
1c: 40 82 00 20 bne- 3c <__libc_recv+0x3c>
20: 38 60 00 0a li r3,10
24: 38 81 00 14 addi r4,r1,20
28: 38 00 00 66 li r0,102
2c: 44 00 00 02 sc
30: 38 21 00 30 addi r1,r1,48
34: 4c a3 00 20 bnslr+
38: 48 00 00 00 b 38 <__libc_recv+0x38>
Again, this is 603e on linux-2.4.16 glibc-2.2.5 gcc-2.95.3. (Odd, I
can't seem to find this function in a statically-linked gdbserver, nor
any reference to it in the gdbserver-6.5 source code).
On the kernel side:
_GLOBAL(DoSyscall)
...
blrl /* Call handler */
.globl ret_from_syscall_1
ret_from_syscall_1:
20: stw r3,RESULT(r1) /* Save result */
li r10,-_LAST_ERRNO
cmpl 0,r3,r10
blt 30f
neg r3,r3
cmpi 0,r3,ERESTARTNOHAND
bne 22f
li r3,EINTR
22: lwz r10,_CCR(r1) /* Set SO bit in CR */
oris r10,r10,0x1000
stw r10,_CCR(r1)
30: stw r3,GPR3(r1) /* Update return value */
b ret_from_except
...
ret_from_except:
...
lwz r3,_CCR(r1)
...
mtcrf 0xFF,r3
...
RFI
Now, I'm a little rusty on PPC asm (I've been doing a lot of ARM
lately), but it looks to me like the kernel is setting bit 0 in CR0
(oris r10, r10, 0x1000) a.k.a LT, but the user side is looking at CR0
(bnslr+) bit 3 a.k.a. SO. Or maybe the other way around, I'm not sure
after reading Sections 1.2 and 2.1 of the Programming Environments manual.
Or am I misinterpreting something? I must be, this is well-trodden code
I'm thinking...
The readchar() in gdbserver's remote-utils.c just calls read() on the
file descriptor for the socket. Still trying to track that code down...
b.g.
--
Bill Gatliff
bgat@billgatliff.com