Can't mmap the top 1MB of /dev/mem?

Hi Dan (and all),

I've been playing w/ mmap for flash and nvram access on the rpxlite(MV
derived kernel). One thing I've noticed is that it won't let me mmap the
top 1MB of the address space.  I'm guessing that the region isn't
represented in the MMU stuff or this is a deliberate block in the /dev/mem
driver.

I've done a little digging on my own and haven't found the cause.  Can
anyone point me to the right file/function(s)?

Many Thanks,
-Mark

Mark S. Mathews

AbsoluteValue Systems      Web:    http://www.linux-wlan.com
P.O. Box 941149            e-mail: mark@linux-wlan.com
Maitland, FL 32794-1149    Phone:  407.644.8582
USA                        Fax:    407.539.1294


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

"Mark S. Mathews" wrote:

...... One thing I've noticed is that it won't let me mmap the
top 1MB of the address space.

That should work, I do this quite regularly.  What actually fails,
the mmap() or your access to the mapped region?

There is what I do:

	mem_addr = (u_char *)mmap(NULL, FLASH_MEM_SIZE,
			(PROT_READ | PROT_WRITE), MAP_SHARED,
			mem_fd, FLASH_MEM_ADDR);

#define FLASH_MEM_SIZE and FLASH_MEM_ADDR accordingly.  I hope there
isn't some weird arithmetic rounding problem when we hit the top.  I
just usually tell this to map the upper 8M bytes.

The MMU doesn't care, if the generic Linux VM subsystem creates the PTEs
it just loads them.  I don't remember any code in the mem driver that
would prevent this either.  None of this is unique to the 8xx.


	-- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Hello, Mark!

Hint: if you are going to program the flash you don't want to allocate the
amount of memory larger than the file you are going to program. And you
probably don't want to erase the initial loader ;-)

But I agree that the kernel should allow you do that and it doesn't.

Regards,
Pavel Roskin


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

On Thu, 22 Jun 2000, Mark S. Mathews wrote:

Hi Dan (and all),

I've been playing w/ mmap for flash and nvram access on the rpxlite(MV
derived kernel). One thing I've noticed is that it won't let me mmap the
top 1MB of the address space.  I'm guessing that the region isn't
represented in the MMU stuff or this is a deliberate block in the /dev/mem
driver.

I've done a little digging on my own and haven't found the cause.  Can
anyone point me to the right file/function(s)?

include/asm/processor.h: -------------------------------------------------

...
#define TASK_SIZE    (0x80000000UL)
...

mm/mmap.c: ---------------------------------------------------------------

...
unsigned long do_mmap(struct file * file, unsigned long addr, unsigned
long len,
        unsigned long prot, unsigned long flags, unsigned long off)
{
...
        if (len > TASK_SIZE || addr > TASK_SIZE-len)
                return -EINVAL;
...


-------------------------------------------------------------------------

Wouldn't this account for it?  I'm not sure where your top 1M of
address space is on your board, but the DY4 board I'm working on
has flash up around 0xfe000000. And we ran into the same problem trying
to map our flash.

We solved the problem in a roundabout way by doing what we need to do
from inside the kernel, calling remap_page_range() directly like
mmap_mem() [drivers/char/mmap.c] does.


Frank.

-----
Frank Smith, MCompSci
Principal Software Designer      frank.smith@amirix.com
AMIRIX Systems Inc.              http://www.amirix.com/
Embedded Debian Project          http://www.emdebian.org/
77 Chain Lake Drive              902-450-1700 x289 (Phone)
Halifax, N.S. B3S 1E1            902-450-1704 (FAX)


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Frank Smith wrote:

Wouldn't this account for it?

No.  Those are virtual address calculations in that file........


	-- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Hello, Dan!

That should work, I do this quite regularly.  What actually fails,
the mmap() or your access to the mapped region?

mmap fails with EINVAL.

There is what I do:

	mem_addr = (u_char *)mmap(NULL, FLASH_MEM_SIZE,
			(PROT_READ | PROT_WRITE), MAP_SHARED,
			mem_fd, FLASH_MEM_ADDR);

I know that program. FLASH_MEM_SIZE is a small number used only in the
first mmap() to determine the size of the flash. If you make it so big
that (int)(FLASH_MEM_ADDR+FLASH_MEM_SIZE)==0 then mmap() fails.

The test program below works on LinuxPPC (2.2.14 with tweaks for G3) but
fails on RPX/Lite (MontaVista kernel):

mmap at 0xffc00000, size 0x400000 failed: Invalid argument

Regards,
Pavel Roskin

=================================
#include <sys/types.h>
#include <sys/mman.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <errno.h>
#define FLASH_MEM_ADDR	((uint)0xffc00000)
#define FLASH_MEM_SIZE	((uint)0x00400000)
int
main(void)
{
	int	mem_fd, i;
	uint	mfg, dev;
	void* mem_addr = NULL;

	if ((mem_fd = open("/dev/mem", O_RDWR)) < 0) {
		perror("cannot open /dev/mem");
		exit(1);
	}

	mem_addr = (u_int *)mmap(NULL, FLASH_MEM_SIZE,
				(PROT_READ | PROT_WRITE), MAP_SHARED,
				mem_fd, FLASH_MEM_ADDR);

	if ((int)mem_addr < 0) {
		fprintf(stderr, "mmap at 0x%x, size 0x%x failed: %s\n",
			FLASH_MEM_ADDR, FLASH_MEM_SIZE, strerror(errno));
		exit(1);
	}
}
=================================


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Pavel Roskin wrote:

I know that program. FLASH_MEM_SIZE is a small number used only in the
first mmap() to determine the size of the flash.

Yes, but then it re-maps based upon the flash devices found....but I'll
bet I don't map the top boot sector.......

..... If you make it so big
that (int)(FLASH_MEM_ADDR+FLASH_MEM_SIZE)==0 then mmap() fails.

So, this should be a pretty obvious boundary condition bug in
drivers/char/mem.c or one of the Linux MM functions.  You want to
learn about MMUs?  Go check some of those files for inspiration......


	-- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Sorry, I should have been more specific....aren't you all mind-readers?
;-)

The mmap is failing w/ errno=EINVAL.  Here's the call:

mem_fd = open("/dev/mem", ORDWR);

p = mmap( NULL, (16*1024*1024),
          (PROT_READ|PROT_WRITE), MAP_SHARED, mem_fd, 0xff000000UL);

15MB works just fine.  I just tried a test w/ offset 0xff800000 and size
of 8MB.  That failed too.  Here's a couple of other tests I just tried:

offset      len        result
-----------------------------------
0xfff00000, 1MB        failed
0xffe00000, 1MB        success

So it looks like it's specifically related to the top 1MB, not the size.

BTW: Dan, I think you may have run into this before.  The sample code you
once sent me only mmaps the lower 7MB of an 8MB flash setup.

This whole thing isn't really a critical issue for me.  More of a
curiosity.  I'm not using those sectors anyway.  ;-)

If it _were_ extremely  important I could always trace the mmap system
call.
_That's_ why Linux is so cool.

Thanks to everyone for the responses,
-Mark

On Thu, 22 Jun 2000, Dan Malek wrote:

"Mark S. Mathews" wrote:

quoted

...... One thing I've noticed is that it won't let me mmap the
top 1MB of the address space.

That should work, I do this quite regularly.  What actually fails,
the mmap() or your access to the mapped region?

There is what I do:

	mem_addr = (u_char *)mmap(NULL, FLASH_MEM_SIZE,
			(PROT_READ | PROT_WRITE), MAP_SHARED,
			mem_fd, FLASH_MEM_ADDR);

#define FLASH_MEM_SIZE and FLASH_MEM_ADDR accordingly.  I hope there
isn't some weird arithmetic rounding problem when we hit the top.  I
just usually tell this to map the upper 8M bytes.

The MMU doesn't care, if the generic Linux VM subsystem creates the PTEs
it just loads them.  I don't remember any code in the mem driver that
would prevent this either.  None of this is unique to the 8xx.


	-- Dan

Mark S. Mathews

AbsoluteValue Systems      Web:    http://www.linux-wlan.com
P.O. Box 941149            e-mail: mark@linux-wlan.com
Maitland, FL 32794-1149    Phone:  407.644.8582
USA                        Fax:    407.539.1294


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

On Thu, 22 Jun 2000, Dan Malek wrote:

Frank Smith wrote:

quoted

Wouldn't this account for it?

No.  Those are virtual address calculations in that file........

Oops. Upon thinking about this a bit more, the problem we were having
was that we were trying to get physical 0xfe000000 mapped to virtual
0xfe000000.

The board we're using has firmware that we want to be able to call
after Linux is up and running. The FW runs from 0x3000-0x60000, but
refers to data up in 0xfe space. Our first naive attempts were to
use mmap from userland, but we ended up putting stuff in the kernel
and mapping flash directly with remap_page_range().


Frank.


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Hello, Mark!

If it _were_ extremely  important I could always trace the mmap system
call.

I already know that it happens in do_mmap(), at this point:

/* offset overflow? */
if (off + len < off)
        return -EINVAL;

I'm puzzled why it doesn't happen on G3.

_That's_ why Linux is so cool.

Yup.

Pavel


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

I figured it was something like that.  Shouldn't that expression be
testing the topmost address of the mmap'd region instead of one beyond it?

i.e.:

   if ( (off+len-1) < off )
	return EINVAL;


It works on the G3?  Perhaps the compiler/processor difference is getting
you into some of the 64-bit register goofiness or a slightly different
code sequence that does something different w/ the overflow flag.  Just
guessing here.

-Mark


On Thu, 22 Jun 2000, Pavel Roskin wrote:

Hello, Mark!

quoted

If it _were_ extremely  important I could always trace the mmap system
call.

I already know that it happens in do_mmap(), at this point:

/* offset overflow? */
if (off + len < off)
        return -EINVAL;

I'm puzzled why it doesn't happen on G3.

quoted

_That's_ why Linux is so cool.

Yup.

Pavel

Mark S. Mathews

AbsoluteValue Systems      Web:    http://www.linux-wlan.com
P.O. Box 941149            e-mail: mark@linux-wlan.com
Maitland, FL 32794-1149    Phone:  407.644.8582
USA                        Fax:    407.539.1294


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

So, this should be a pretty obvious boundary condition bug in
drivers/char/mem.c or one of the Linux MM functions.  You want to
learn about MMUs?  Go check some of those files for inspiration......

Already. This should be The Right Thing. Anybody wants to send it to
Linus?

len is not 0, it is clear from the code above. (off + len) points to the
byte that is after the allocated area, (off + len - 1) points to the
last byte. The last byte should not be on the "other end of memory"

=======================
diff -u -r1.1.1.1 mmap.c

--- mm/mmap.c	2000/01/03 20:11:04	1.1.1.1
+++ mm/mmap.c	2000/06/22 19:52:51

@@ -186,7 +186,7 @@
 		return -EINVAL;

 	/* offset overflow? */
-	if (off + len < off)
+	if (off + len - 1 < off)
 		return -EINVAL;

 	/* Too many mappings? */

=======================

Regards,
Pavel Roskin


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Hello, Mark!

   if ( (off+len-1) < off )
	return EINVAL;

It's done this way in Linux-2.2.14 :-)

It works on the G3?  Perhaps the compiler/processor difference is getting
you into some of the 64-bit register goofiness or a slightly different
code sequence that does something different w/ the overflow flag.  Just
guessing here.

Absolutely no "64-bit register goofiness" :-)))

Pavel


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Pavel Roskin wrote:

Already. This should be The Right Thing. Anybody wants to send it to
Linus?

I'll try to sneak it in.  We are already "discussing" some of my
other changes :-).

There you go Mark.  Patch it and try it.

Thanks.


	-- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/