Recently I came back to look into coreboot. Mainly because low level is fun and skills related to firmware (even coreboot) starting get attention on freelance portals (first odesk job, second odesk job). I was surprised that under the wings of Google coreboot team start to support ARM (BTW ARM programming is IMHO next great skill to learn). So I cloned latest, code compiled QEMU armv7 mainboard model and tried to kick it in latest qemu-system-arm. Unfortunately it didn’t boot. Below you can find my TL;DR debugging story.
coreboot qemu-armv7 mainboard compilation – very quick steps
1 2 3 4
Mainboard -> Mainboard model -> QEMU armv7 (vexpress-a9)
NOTE: To prevent annoying warning about XML when running gdb from coreboot crossgcc utilities:
libexpat1-dev should be installed.
1 2 3 4 5
buildgcc will provide armv7 toolchain with debugger (
-G) and compilation
will use 8 parallel jobs.
qemu-system-arm compilation – very quick steps
1 2 3 4 5
Use good gdbinit, so with every instruction executed gdb will automatically
provide most useful informations. IMHO good choice is
fG! gdbinit shared on
github. It contain support for ARM and
x86. To switch to ARM mode inside gdb simple use
arm command. Output looks
Noob dead end
Command for running qemu that I found in early qemu-armv7 commit log:
It ends with qemu error:
1 2 3 4 5 6 7 8
At the beginning I thought that it is a mistake so I tried:
What ends with:
1 2 3 4 5 6 7
Obviously qemu complains on value in R15 (PC – Program Counter), which is the address of current instruction (like EIP in x86).
Stepping through assembler instructions using cross-compiled debugger
util/crossgcc/xgcc/bin/armv7-a-eabi-gdb) points to:
ldmia will load from stack values of all given registers. This cause that PC
goes to 0x0 and then run instruction from zeroed memory, which in ARM
It happens till PC reach 0x4000000 which is out of ‘RAM or ROM’ for qemu.
Unfortunately there is no sign about
ldmia instruction with above range of
registers in coreboot and qemu code.
I knew that at some point qemu worked with coreboot. I tried few versions and
it leads me to some commit between
switch I was able to narrow down problem to one commit that change
VE_NORFLASHALIAS option for vexpress-a9 to 0
It looks like for vexpress-a9 qemu place kernel at 0x60000000
(vexpress.highmem), which is aliased to range 0x0-0x3ffffff.
VE_NORFLASHALIAS=0 cause mapping of vexpress.flash0 to the same region as
kernel and because flash (
-bios) was not added we have empty space (all
zeros) what gives
andeq r0, r0, r0.
Right now I have working version of coreboot but only with
VE_NORFLASHALIAS=-1 set in hw/arm/vexpress.c. The main questions are:
- what is the correct memory map for qemu-armv7 and how coreboot should be mapped ?
- what’s going on with coreboot or qemu that I can’t go through bootblock ?
I tried to debug coreboot executed from flash:
Coreboot as UEFI has few phases. For UEFI we distinguish SEC, PEI, DXE and BDS (there are also TSL, RT and AL, but not important for this considerations). On coreboot side we have bootblock, romstage, ramstage and payload.
qemu-armv7 bootblock failure
qemu-armv7 booting procedure start from
_rom section which contain hardcoded
reset procedure. After that go through few methods like on below flow:
1 2 3 4 5 6 7 8 9
At the end of
dcache_foreach we experience failure because
instruction tries to restore registers from stack, which should be stored at
the beginning of
Unfortunately for some reason stack doesn’t contain any reasonable values (all
stmdb. Why is that ?
Obvious things are not so obvious
As I point above everything seems to be related with memory map for
vexpress-a9. I wrote question to qemu developers mailing list describing all
the problems. You can read it
So the answer is that ARM Versatile Express boards in general have two
different memory maps. First is legacy with RAM in low memory and second is
modern with flash in low memory instead of RAM. Since qemu
memory map was used. That’s why I saw change in behavior. Obviously flash in
qemu is read only, so no matter what pushing on stack didn’t work.
coreboot stack location fix
I though that fix would be easy. One thing that I have to do is change stack address. The question is where to place the stack ? So I took a look at qemu memory map:
1 2 3 4 5 6 7 8
SRAM is temporary storage where I decide to put stack. The change in coreboot looks like below:
1 2 3 4 5 6 7 8 9 10 11
I changed STACK_TOP and STACK_BOTTOM.
Unfortunately still I was unable to boot coreboot on vexpress-a9. Situation
improved because stack start to work correctly and accept push and pop data
to/from, but next problem occurs in
As CBFS specification explains:
CBFS is a scheme for managing independent chunks of data in a system ROM.
Default CBFS media initialization for qemu-armv7 leads to
init_emu_rom_cbfs_media that fills
cbfs_media structures with function
pointers that help to operate on CBFS.
1 2 3 4 5 6 7 8 9
The problem was that pointers were relative to bootblock base address
-bios option maps coreboot.rom from address
leads to change in bootblock base address to
1 2 3
This solve other issue not mentioned till now. I didn’t know why I can’t load
symbols for bootblock using
add-symbol-file in gdb. Of course reason was
bootblock didn’t start at 0x0 but at 0x10000. Since this moment I could debug
bootblock using lines of C code, by simply:
It was not the end because another error popped up:
memcpy during CBFS decompression
Problem was with storing registers
stmia during memcpy. Backtrace:
1 2 3 4
For some reason R0 (to which we store), contain strange address 0x10000. No
value was stored in this memory range, because again it was read only flash.
Address is passed from upper layers –
cbfs_get_file_content. During debugging
I realize that this address means
ROMSTAGE_BASE. So I changed
to somewhere in SRAM.
1 2 3
What I saw when trying to boot coreboot with this fix was wonderful log proved that coreboot boots without problems.
Above debugging session was all about memory map. It was really fun to experience all those issues because I had to understand lot of ARM assembly instructions, track memory, read the spec, read coreboot and qemu code. It gave me a lot of good experience. If you have any questions or comments please let me know. And finally what is most important it was next thing done on my list.
I think next challenge could be experiment with Linux kernel booting. Coreboot can boot kernel directly or through payload with bootloader.
Thanks for reading.