identity mapped paging (Vaibhav Jain)

-----Original Message-----
From: Vaibhav Jain [mailto:vjoss197 at gmail.com]
Sent: Wednesday, April 18, 2012 3:49 AM
To: Pranay Kumar Srivastava
Cc: kernelnewbies at kernelnewbies.org
Subject: Re: identity mapped paging (Vaibhav Jain)

On Tue, Apr 17, 2012 at 3:46 AM, Pranay Kumar Srivastava
[off-list ref] wrote:

quoted

-----Original Message-----
From: Vaibhav Jain [mailto:vjoss197 at gmail.com]
Sent: Tuesday, April 17, 2012 4:07 PM
To: Pranay Kumar Srivastava
Cc: kernelnewbies at kernelnewbies.org
Subject: Re: identity mapped paging (Vaibhav Jain)


On Fri, Apr 13, 2012 at 2:15 AM, Vaibhav Jain [off-list ref]
wrote:

quoted

I am not clear about the use of identity mapped paging while paging

is

quoted

being enabled by the operating system. Also I don't understand at

what

quoted

point are the
identity mappings no longer useful.According to this article
http://geezer.osdevbrasil.net/osd/mem/index.htm#identity - "The

page

quoted

quoted

table
entries used to identity-map kernel memory can be deleted once

paging

quoted

quoted

and
virtual addresses are enabled." Can somebody please explain?

Identity mapping is when VA(Virt Address)=PA(Physical address).

So basically when you set up your page tables you need to make sure
they map identically. This is very easily done if you consider each

4KB

quoted

block as a page beginning from location 0 upto whatever you've found

to

quoted

be the highest memory available either thru BIOS or GRUB.

Remember that while setting up your PTEs and PDE every address is a
physical one. So if you thought that your kernel would be linked
initially to a higher VA since you would remap it to a lower memory
physically then that would be WRONG!. Without PTEs and PDEs installed
don't do that!.

Why would you want it? Well for a simple reason, when your kernel
starts to boot there's no translator,(No PTEs/PDEs and the Paging
Enabled bit of processor is also cleared AFAIK just after the BIOS is
done), yet since you've not enabled your processor for that but

you'll

quoted

be doing that in a moment.

So let's say you made your kernel to be linked to higher VA like

3Gigs.

quoted

Now the addresses would be generated beginning 3Gigs however you

still

quoted

don't have the Page tables installed since your kernel just started.

So

quoted

in that case the address is the physical address. And if you've not
loaded your kernel beginning 3Gigs then it would definitely come
crashing down.

To avoid the crash in case you made your kernel to link to higher

half

quoted

of the memory, you can use GDT trick since segmentation is always on
and you can make the overflow of the address addition to translate to

a

quoted

lower physical memory even if paging is not enabled yet. Thus it is
possible to load the kernel at lower memory addresses while the

linkage

quoted

would be for higher VMA. And once your PTEs/PGD are enabled then you
can use those instead of the GDT trick.

Here's a link to that http://wiki.osdev.org/Higher_Half_With_GDT

quoted

Thanks
Vaibhav Jain

Hi,

Thanks for replying but I am still confused. I continued reading

about

quoted

this thing and what
I have understood is the following :
After the kernel executes the instruction to enable paging the
instruction pointer will contain the
address of the next instruction which will now be treated as a

virtual

quoted

address. So for the next instruction to be executed
the page table should map this address to itself.
Please correct me if I am wrong.
I am confused by the point about linking  the kernel to higher

address.

quoted

Could you please put that in a step by step manner
to make it clear what  happens before paging is enabled and what
happens after that.
Also, please explain at what point during the execution of kernel

code

quoted

are the identity-mapped addresses no longer useful ?




Thanks
Vaibhav
Hi,

I am somewhat understanding your point. But I have some other queries
now in my mind.

If the kernel is linked to 3Gigs is there a way other than the GDT
trick.?

Make your load address = VA when you link so you won't have to worry
about doing the GDT trick.

quoted

In fact I am wondering that if the kernel is linked to 3Gigs and Grub
loads it at 1MB physical, how will even the first instruction of

kernel

quoted

execute ?  I mean if all the address generated by kernel are above 3
Gigs and paging is not enabled how will it start
running ?

That's what the GDT trick is for. If you read the intel/amd processor
manuals the segmentation is always on. So when the address get
generated your segment's base address is still added to the generated
address before it is put on wire. You can add a constant offset (in
your GDT's base address part) to the generated address to get the
address beginning from the load address of your kernel.

I would suggest you make the higher half kernel later and try to first
create some code that can fragment your available memory into pages and
store this information so you'll know what all pages are there. Next
would be to do identity mapping, since your kernel VMA=LMA in your
linker script this would be easier to do.

When you get that paging enabled you can move on to higher half kernel.
I would suggest you to work on page replacement algos and virtual
memory management code side by side for better integration with paging
in later stages.

Maybe you can post your code if you are allowed to then I can have a
look at it.

quoted

Thanks
Vaibhav Jain

-----------------------------------------------------------------------
-----------------------------------------------------------------------
-------------------


Thanks for the explanation!!  Please confirm what I have concluded :
- If the kernel is linked to the same address it is loaded at ,
identiy-mapping is required so that all the addresses generated by the
kernel map to correct memory.

- If the kernel is NOT loaded at  the same address it is linked to (for
e.g. linked to 3 Gigs and loaded at 1MB) identity mapping is not
required.

Yes and No. Yes if you don't have paging enabled, NO if you will be enabling paging but you haven't mapped your physical
Memory yet to the addresses. Like I said before make a memory management routine before you enable paging, trust me you'll save a hell lot of time in later stages if you are really serious about making your kernel work.

- if the kernel is linked AND loaded to 1MB, then to move to higher
half kernel two types of page mappings will be required -

a) identity-mapping of lower addresses
b) mapping all the virutal addresses above 3Gigs  to lower adresses

Yes and No. Yes when initially you setup paging, the paging code would itself be in the process of being setup so most probably you would require identity paging here if you are not using GDT trick.
No when the rest of your kernel, i.e. after enabling paging, is being loaded such that it is able to work if its linked to higher VMA. This is since you already have got your page tables up and initialized.

What you can try here is split your kernel in two halves, one that just enables enough paging for you to work with initially. The rest of it you can even have as an executable and have code that loads it properly.

Actually I am going through a tutorial to write a kernel. In the
tutorial the linker script sets the location counter to 1MB and the
code is working fine.

But I looked at some other tutorials and osdev articles which mention
about higher half kernel and linking to 3Gigs and got confused.So I
wanted to
be clear about the concepts before I make any changes to the code.

Osdev articles are very good but you should understand linker scripts and try identity paging first. Make your code kernel code page fault by writing in some memory location not mapped and check if you are able to handle that. After that move to higher half kernel, it would be best though to split your kernel at that point since it would become too big to load I guess by grub, but maybe your disk driver can do that?

 Thanks
Vaibhav Jain

::DISCLAIMER::
-----------------------------------------------------------------------------------------------------------------------

The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only.
It shall not attach any liability on the originator or HCL or its affiliates. Any views or opinions presented in
this email are solely those of the author and may not necessarily reflect the opinions of HCL or its affiliates.
Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of
this message without the prior written consent of the author of this e-mail is strictly prohibited. If you have
received this email in error please delete it and notify the sender immediately. Before opening any mail and
attachments please check them for viruses and defect.

-----------------------------------------------------------------------------------------------------------------------