From: Milton Miller <hidden> Date: 2008-07-17 16:12:26
On Thu Jul 17 at 23:22:28 EST in 2008, Guillaume Dargaud wrote:
Hello all,
I'm in the process of writing a mini-bootloaler for a custom board and
would
like some feedback on my boot methodology.
Basically the kernel code (elf file) is copied into memory by a JTAG
debugger. A custom program (yet to be finished) then copies it onto an
onboard flash memory using serial commands. This flash is not part of
the
flat memory addressing.
When I want to boot, I have a small bootloader put into an eprom that
_is_
part of the flat memory addressing. This (finished but not tested yet)
bootloader basically does two things:
- it copies the content of the first flash into RAM
- it launches it using a custom function call like "0x400000();"
Now, I'd like to know if this is a reasonable approach ? I couldn't
think of
a better way, but maybe there are issues I didn't think off.
This is a very reasonable approach, and is quite similar to what
I do.
You actually have a few choices. You can put just the loaded
data, or you can put the elf file and parse the header in your
boot loader. After seeing the advantages, I would keep the elf
header. One big advantage of keeping the elf header is you can
see how much data will be zeroed for bss when the kernel starts.
Another choice is rather than loading the kernel, build a zImage
of some kind (see the code in arch/powerpc/boot). This way the
kernel code and static data is compressed. I often see a 3-fold
reduction in size. You can also attach an initrd, should you
decide to use initramfs later. More o this below.
Both the kernel and zImage will build an elf file with a single
load header. Both can run regardless of where they are loaded,
so you don't have to worry about parsing the load address, just
calculating the relatve entry point. (The kernel will move
itself down to address 0 (except book-e parts), and the zImage
will run where it is started. However, the default zImage will
fail to start the kernel if it isn't loaded above the link size
of the kernel (including bss).
One additional question, if the above is valid, is that I would like
to pass
a particular argument to the kernel to set its MAC address. I can read
an
external value (similar to jumpers) from the bootloader via GPIO. Is
there a
way to pass this value to the kernel so that a modified
arch/ppc/boot/simple/embed_config.c can use it to set the MAC address ?
Maybe the main() of the kernel can receive argv/argc the usual way,
and I
just need to call "0x400000(argc, argv);" but I have no idea if that
works.
Ahh ... you are still on ppc. Please note that we just merged
the removal of arch/ppc, everyone needs to use powerpc now. The
good news is: its easier to state the requirements, and its
easier to share the code in vmlinux. The bad news is you have to
follow the rules for passing data to the kernel. Its not hard.
We have defined a data structure that is parsed to become a tree
of data describing the machine, and based the contents on open
firmware. We call this the flattened device tree.
To pass the MAC address, one uses a property to the device node
which also describes what the hardware is (by name) and the
address space it uses. Similarly, the kernel finds out how fast
the cpu and timebase are running by reading properties in the cpu
node.
Please read Documentation/powerpc/booting-without-of for more
information. To summarize, the kernel is started with a pointer
to the device tree structure, and it contains all the information
needed. All the data the kernel needs will be in one flat linear
chunk, but the kernel will move itself in memory to its linked
location. The 32 bit kernel does require r1 to be set to the
top of a memory block that can be used as a stack.
There are several options to get the data into the device tree.
If you choose to use the zImage format mentioned above, the
device tree library is compiled in and you would just write a bit
of c code to read your structure and tell the library to put it
in the device tree. The standard initialization preserves r3-r6
so you get 4 32 bit args (or maybe 5) to pass data or pointers.
Ohterwise, there is a tool called dtc (device tree compiler) that
that creates the binary structure from a source file that is
human readable. From this, there is a library that will modify
the tree at runtime.
Another approach is to modify the device tree directly from your
first stage. While you can link the library into your first
stage, you can also just tell dtc to output an assembly file with
symbolic labels on the data you need to modify. You would then
pass this structure to the zImage wrapper or directly to the
kernel.
To see how simple this approach can be when you only need to
modify a few parameters, I will point out my [qemu-rom v2 patch.
Hmm, thats not it, where is it? I thought I had posted this to
linuxppc-dev but can't find it.]
I will point out the minimal rom image I did for qemu where the
device tree is linked into the assembly, the kernel elf file is
already loaded, but I had to copy the nvram data serially into
ram and then poke the memory size and initrd location into the
device tree.
http://landley.net/hg/firmware/raw-file/92f89c9c9495/sources/toys/make-
ppc_rom.tar.bz2
From: David Gibson <hidden> Date: 2008-07-18 03:47:27
On Thu, Jul 17, 2008 at 11:14:11AM -0500, Milton Miller wrote:
On Thu Jul 17 at 23:22:28 EST in 2008, Guillaume Dargaud wrote:
[snip]
quoted
Maybe the main() of the kernel can receive argv/argc the usual way,
and I
just need to call "0x400000(argc, argv);" but I have no idea if that
works.
Ahh ... you are still on ppc. Please note that we just merged
the removal of arch/ppc, everyone needs to use powerpc now. The
good news is: its easier to state the requirements, and its
easier to share the code in vmlinux. The bad news is you have to
follow the rules for passing data to the kernel. Its not hard.
We have defined a data structure that is parsed to become a tree
of data describing the machine, and based the contents on open
firmware. We call this the flattened device tree.
Your firmware doesn't have to deal with this though, although it can.
It's perfectly acceptable - in fact I'd mildly recommend it over doing
this in the firmware - to instead have the kernel's zImage wrapper
supply the flattened device tree, folding information it needs to get
from the firmware into it.
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
From: Milton Miller <hidden> Date: 2008-07-18 17:45:26
On Jul 17, 2008, at 10:47 PM, David Gibson wrote:
On Thu, Jul 17, 2008 at 11:14:11AM -0500, Milton Miller wrote:
quoted
On Thu Jul 17 at 23:22:28 EST in 2008, Guillaume Dargaud wrote:
[snip]
[more snip]
quoted
We call this the flattened device tree.
Your firmware doesn't have to deal with this though, although it can.
It's perfectly acceptable - in fact I'd mildly recommend it over doing
this in the firmware - to instead have the kernel's zImage wrapper
supply the flattened device tree, folding information it needs to get
from the firmware into it.
I thought about adding a section on which to recommend or talk about
the tradeoffs, but got hungry and hit send before going to lunch.
Yes, I the zImage choice is definitely the least amount of code to
write, and is a good tradeoff to get future upgradability. It may not
be the absolute fastest split of the work to start the kernel but it is
reasonably good, especially as it will decompress the kernel to its
run-time destination.
milton
This is a very reasonable approach, and is quite similar to what
I do.
Makes me feel better !!!
You actually have a few choices. You can put just the loaded
data, or you can put the elf file and parse the header in your
boot loader.
I cannot do that for memory constrains. I have 4Mb in the flash for the
kernel and only a few Kb for the eprom of the bootloader. Also the idea is
to sometimes update the kernel but basically never the bootloader.
After seeing the advantages, I would keep the elf
header. One big advantage of keeping the elf header is you can
see how much data will be zeroed for bss when the kernel starts.
Another choice is rather than loading the kernel, build a zImage
of some kind (see the code in arch/powerpc/boot). This way the
kernel code and static data is compressed. I often see a 3-fold
reduction in size. You can also attach an initrd, should you
decide to use initramfs later. More o this below.
I'm not too familiar with the sequence of events that unfolds when the
kernel start.
The JTAG debugger copies the kernel at a given address. A program dumps a
copy of that address range in flash. A bootloader does the reverse and
launches it, so it shouldn't really matter what form the kernel takes, as
long as the original _did_ work.
Ahh ... you are still on ppc. Please note that we just merged
the removal of arch/ppc, everyone needs to use powerpc now. The
Yes, my code works on the ML405, now I'm just trying to get it to work on
our custom board... I haven't been able to follow closely the PPC/PowerPC
change but it got me worried so I stopped updating at 2.6.15. I'm using the
Xilinx git tree because I rely on a lot of their drivers, so I don't know if
the change will be seemless. From the change of xparameters.h to and
entirely different device description method, I'd say I'm not ready to take
the jump.
Any Xilinx user care to comment on that ?
And I'm supposed to release the alpha version of our code today !!!
good news is: its easier to state the requirements, and its
Hmmm...
easier to share the code in vmlinux. The bad news is you have to
follow the rules for passing data to the kernel. Its not hard.
We have defined a data structure that is parsed to become a tree
of data describing the machine, and based the contents on open
firmware. We call this the flattened device tree.
Our firmware is custom VHDL code. I'll go ask the 'tronics guy if it fits
open firmware requirements, but I'd be surprised.
Please read Documentation/powerpc/booting-without-of for more
Will do. Last update is 2005. Is this still relevant ?
I will point out the minimal rom image I did for qemu where the
device tree is linked into the assembly, the kernel elf file is
already loaded, but I had to copy the nvram data serially into
ram and then poke the memory size and initrd location into the
device tree.
Maybe I can just use an elementary approach like that: have the bootloader
write a single byte with the jumper positions somewhere in RAM, and then
have the kernel read it to use for the Mac address. With the flat addressing
it could work if there aren't built-in securities.
Thanks
--
Guillaume Dargaud
http://www.gdargaud.net/
From: Milton Miller <hidden> Date: 2008-07-18 18:36:28
On Fri Jul 18 18:43:11 EST 2008, Guillaume Dargaud wrote:
Hello Milton and David, thanks for the answers.
quoted
This is a very reasonable approach, and is quite similar to what
I do.
Makes me feel better !!!
quoted
You actually have a few choices. You can put just the loaded
data, or you can put the elf file and parse the header in your
boot loader.
I cannot do that for memory constrains. I have 4Mb in the flash for the
kernel and only a few Kb for the eprom of the bootloader. Also the
idea is
to sometimes update the kernel but basically never the bootloader.
Then the zImage approach is definitly the way to go. Actually, dtbImage
now, because you want the device tree to be compiled and linked into it.
(It only took me 33 lines of assembly, and 6 were labels or blank, to
skip over the elf header -- the vmlinux and zImage don't require they be
moved into a specific place - but keeping it just a binary blob is
fine).
quoted
After seeing the advantages, I would keep the elf
header. One big advantage of keeping the elf header is you can
see how much data will be zeroed for bss when the kernel starts.
Another choice is rather than loading the kernel, build a zImage
of some kind (see the code in arch/powerpc/boot). This way the
kernel code and static data is compressed. I often see a 3-fold
reduction in size. You can also attach an initrd, should you
decide to use initramfs later. More o this below.
I'm not too familiar with the sequence of events that unfolds when the
kernel start.
The JTAG debugger copies the kernel at a given address. A program
dumps a
copy of that address range in flash. A bootloader does the reverse and
launches it, so it shouldn't really matter what form the kernel takes,
as
long as the original _did_ work.
Ok, should not be a problem. You will want the wrapper code to create a
flat binary image, so have your platform set the binary variable like
cuboot. Whatever you specify for your platform file will be linked at
the beginning of the image.
When choosing where to laod the zImage into memory, you should consider
that the linked kernel size (including its bss) needs to fit below the
load address, and the extracted device tree and random malloc heap
normally
reside the wrapper.
quoted
Ahh ... you are still on ppc. Please note that we just merged
the removal of arch/ppc, everyone needs to use powerpc now. The
Yes, my code works on the ML405, now I'm just trying to get it to work
on
our custom board... I haven't been able to follow closely the
PPC/PowerPC
change but it got me worried so I stopped updating at 2.6.15. I'm
using the
Xilinx git tree because I rely on a lot of their drivers, so I don't
know if
the change will be seemless. From the change of xparameters.h to and
entirely different device description method, I'd say I'm not ready to
take
the jump.
Any Xilinx user care to comment on that ?
I'm not a user, but from reading the list, they have integrated creating
the dts file into their build system.
And I'm supposed to release the alpha version of our code today !!!
quoted
good news is: its easier to state the requirements, and its
Hmmm...
quoted
easier to share the code in vmlinux. The bad news is you have to
follow the rules for passing data to the kernel. Its not hard.
We have defined a data structure that is parsed to become a tree
of data describing the machine, and based the contents on open
firmware. We call this the flattened device tree.
Our firmware is custom VHDL code. I'll go ask the 'tronics guy if it
fits
open firmware requirements, but I'd be surprised.
I'm sure your firmware does not presently create this tree, but
the dtbImage format is designed to be a delivery vehicle for it.
quoted
Please read Documentation/powerpc/booting-without-of for more
Will do. Last update is 2005. Is this still relevant ?
That's might be the last time we changed the structure of the tree,
but we have been updating the definition of what goes into the tree
structure pretty much every month since then. In the 2.6.27 kernel,
I think we started breaking out these descriptions into the
dts-bindings directory tree.
quoted
I will point out the minimal rom image I did for qemu where the
device tree is linked into the assembly, the kernel elf file is
already loaded, but I had to copy the nvram data serially into
ram and then poke the memory size and initrd location into the
device tree.
Maybe I can just use an elementary approach like that: have the
bootloader
write a single byte with the jumper positions somewhere in RAM, and
then
have the kernel read it to use for the Mac address. With the flat
addressing
it could work if there aren't built-in securities.
We are happy for you to read such information in the zImage (or
dtbImage).
In fact we have a hook "fixups" that is the designated place for this to
happen. If the location to read the jumper is well defined, I would
suggest doing all of it from your fixup function, you don't need to save
the jumper position in ram (less code for your bootloader, and probably
a simpler handoff). (Needing the mac address does mean you can't use
the
simpleboot.c).
Hope you find these comments helpful,
milton
Thank you Milton for the detailed explanation. It'll take me quite a while
to digest it.
As a follow up to my previous messages, I now have a working kernel and a
working bootloader... but not when they are both together.
Case in point:
- load zImage.elf to DRAM 0x400000 with JTAG debugger. Run it. It runs fine.
- Load Bootloader.elf to BRAM 0xFFFF0000 with JTAG debugger. Run it. It runs
fine (but it doesn't do much yet).
- Now load both of the previous ones, and have the bootloader perform a jump
to kernel:
typedef void tFunc(void);
((tFunc *)0x400000)();
The kernel seems to start:
loaded at: 00400000 004EA19C
board data at: 004E8120 004E819C
relocated to: 0040405C 004040D8
zimage at: 00404E48 004E7A7E
avail ram: 004EB000 08000000
Linux/PPC load: console=ttyUL0,115200 rw root=/dev/nfs ip=bootp
Uncompressing Linux...done.
Now booting the kernel
But then it hangs for exactly 3 minutes, nothing on the serial port, before
the bootloader restarts.
Why should the behavior be different whether it's started from the debugger
or from my bootloader ?
--
Guillaume Dargaud
http://www.gdargaud.net/
Answering to myself to try and provide more background info:
As a follow up to my previous messages, I now have a working kernel and a
working bootloader... but not when they are both together.
Case in point:
- load zImage.elf to DRAM 0x400000 with JTAG debugger. Run it. It runs
fine.
- Load Bootloader.elf to BRAM 0xFFFF0000 with JTAG debugger. Run it. It
runs fine (but it doesn't do much yet).
- Now load both of the previous ones, and have the bootloader perform a
jump to kernel:
void main() {
typedef void tFunc(void);
((tFunc *)0x400000)();
}
Now if the bootloader _only_ contains the above code, the kernel runs
perfectly.
Unfortunately I need to do some things in the bootloader: I use 3 GPIOs to
control a frontal set of LEDs, to get a card number and to load/unload a
serial flash memory. Those operations work. But here is the strange thing
that leads me to believe there are interrupt interferences.
If I just read from a GPIO before launching the kernel, it boots fine, with
a few NFS errors when mounting the rootnfs:
[ 4.360060] Freeing unused kernel memory: 76k init
[ 11.432269] nfs: server 192.168.1.185 not responding, still trying
[ 11.441186] nfs: server 192.168.1.185 OK
init started: BusyBox v1.10.1 (2008-04-24 12:26:38 CEST)
genepy login:
Now if I also write to a 2nd GPIO, I get a lot more NFS errors.
And if I also access the flash through the 3rd GPIO, the kernel hangs:
loaded at: 00400000 004EA19C
board data at: 004E8120 004E819C
relocated to: 0040405C 004040D8
zimage at: 00404E48 004E7A7E
avail ram: 004EB000 08000000
Linux/PPC load: console=ttyUL0,115200 rw root=/dev/nfs ip=bootp
Uncompressing Linux...done.
Now booting the kernel
But then it hangs for exactly 3 minutes, nothing on the serial port,
before the bootloader restarts.
Why should the behavior be different whether it's started from the
debugger or from my bootloader ?
Now I've defined my hardware project using interrupts on the GPIOs, which I
don't need and frankly they don't make sense for an output GPIO. I would
like to try to compile a kernel without GPIO interrupts, unfortunately the
compilation fails:
$ make ARCH=ppc CROSS_COMPILE=powerpc-linux-uclibc-
[...]
CC arch/ppc/syslib/virtex_devices.o
arch/ppc/syslib/virtex_devices.c:535: error: 'XPAR_INTC_0_GPIO_0_VEC_ID'
undeclared here (not in a function)
Question breakdown:
- do I need interrupts for GPIOs ?
- how can I compile the ppc 2.6.25 kernel without them ?
- Why do I have problems running the kernel if the bootloader makes use of
the GPIOs ?
- does the bootloader need to perform some kind of special cleanup before
calling the kernel ?
Thanks a lot for shedding a light on those mysteries.
--
Guillaume Dargaud
http://www.gdargaud.net/