Hi Russell,

On Mon, 2009-08-31 at 09:30 +0100, Catalin Marinas wrote: 

On Sat, 2009-08-29 at 13:27 +0100, Catalin Marinas wrote:

quoted

It looks like this is causing the problem. Setting the protection to RW
and writing data (not instructions) causes the text page to be COW'ed
(page mapped with MAP_PRIVATE). Some cache flushing is missing on VIPT
caches during page copying for COW. With ARM11MPCore, the D-cache is
write-allocate so it never makes it to the main memory for the I-cache
to pick.

I'll look again next week on where to best add the flushing (or just
modify the dynamic linker to avoid COW on text pages). Any suggestions?

After talking to the toolchain people, it seems that the dynamic linker
is just doing whatever the ELF file says regarding the relocations. The
problem in this case is that when compiling with -pie, one of the crt*.o
files (and _start) used in PIE applications is not position-independent.

I think this was fixed (but not released yet) by CodeSourcery but you
can get this behaviour if some files of an executable were not compiled
with -fpic. So the mprotect cache flushing patch that I posted looks
like a valid workaround.

There is a glibc patch to allow fully position-independent code and
avoid CoW for text pages:

http://sourceware.org/ml/libc-ports/2008-10/msg00009.html

We tested it and it seems to solve the problem without requiring a
kernel patch.

Any opinion on whether the ARM kernel should support dynamic shared
objects where not all objects are position-independent? IOW, whether
text relocations are allowed to be resolved at run-time rather than
compile (static link) time for the dynamic shared objects? AFAICT, there
isn't anything in the ARM EABI which would prevent this, so a kernel
patch may be needed.

There is a similar situation for RWX pages and CoW. In this case, the
copy_user_highpage() function should probably be modified to clean the
D-cache and invalidate the I-cache.

Thanks.

-- 
Catalin