Boot flow
kernel/src/arch/*/ + main.rs Β· The three boot protocols, early memory, kernel image layout, SMP bring-up, trap/syscall entry and the device-address discipline Β· Sources: docs/architecture/boot.md, riscv64.md, aarch64.md, design-divergences.md (D-17/D-18)
Β§1Boot invariants
All three architectures share an explicitly recorded set of boot invariants (boot.md):
- Pre-allocator code uses static BSS pools only β every early-stage data structure is self-sufficient without a heap;
- The boot CPU owns all early initialization; secondaries spin on a release flag until the scheduler exists;
- The init= resolution and the PID-1 contract sit at the end of rust_main β the ltpscan and lmbench runners are themselves PID-1 ELFs.
Generic sequence (all three ports): early arch bring-up (page tables, serial) β memory init (memblock, zones, vmemmap, the small-object allocator) β traps/IRQs β scheduler and init task β driver probe β mount root β run init.
Β§2riscv64: OpenSBI MβS
- Boot chain: QEMU loads OpenSBI (fw_dynamic) in M-mode and hands the S-mode kernel the entry at
_start(a0 = hartid, a1 = DTB pointer); boot.S sets the stack, clears BSS PC-relative, builds the trampoline page table (first 2 MB kernel VAβPA) then the full early page table, enables the MMU and jumps to VA to enter rust_main. - SMP bring-up: OpenSBI HSM starts all harts; the first to reach init() via CAS becomes the boot hart; the rest wait on
SMP_INIT_DONEwith wfi. - The R87 secondary bring-up window (lesson): secondaries once waited with the timer already enabled + wfi β during the whole BSP tail (fs mounts, drivers, init spawn; scales with -m), every tick could exit through irq_exit preemption into __schedule() while PER_CPU[hart] was still a zeroed slot; the zero slot triggered heal chains, the hart executed enqueued tasks while holding its boot stack, and slot fights booked the wrong prev at a context switch β mixed-context save (one task's registers written into another's thread record), detonating as the "kill the idle task" KERNPANIC. Fix: spin (not wfi β QEMU's wfi only wakes on an ENABLED pending interrupt) until is_boot_complete, and enable timer/external/software interrupts only AFTER init_secondary publishes PER_CPU[hart]. Companion finding: boot.S zeroes sie on every hart while ipi::init only sets SSIE on the boot hart β without per-hart SSIE, a secondary ran with all IPIs masked (resched/TLB/call-function pending forever; tick-quantized futex wakeβrun 9118 Β΅s vs 24 Β΅s same-hart). The rule on record: a secondary hart takes no interrupts until its per-CPU state is fully initialized and it has entered the idle loop (Linux's own rule).
- Per-CPU interrupt stacks: 16 KB per CPU, 16-byte aligned; used for kernel-mode traps (user traps use task kernel stacks).
Β§3x86_64: multiboot1 + bzImage dual protocol
- The multiboot1 path: QEMU
-kernelELF, entered in 32-bit protected mode (eax = 0x2BADB002, ebx = the multiboot info pointer); boot.S saves the pointer, builds the early page tables (identity low + higher-half kernel map) and enables long mode; the multiboot blob lives in low memory β everything needed is copied into kernel BSS immediately (memory map, cmdline, bootloader name), because the bootstrap identity mapping dies once the real page tables install. - The bzImage path (Linux boot-protocol header + 16-bit trampoline): 16-bit β protected mode (hardcoded absolute far jumps) β 0x100000 (the 32-bit boot stub) β long mode β kernel; the same image boots where a Linux-boot-protocol loader is expected.
- Lifecycle of the bootstrap identity map: the bootstrap PML4 carries the low identity window (serving only the boot stack); after
mm::init()builds the real higher-half tables and switches CR3, PML4[0] is UNLINKED immediately after the stack switch (drop_bootstrap_identity) β no low identity window exists at runtime (D-17, Β§7). - Above-4G BARs: the host bridge may place virtio 64-bit BARs at arbitrary above-4G addresses β two static alias windows plus a dynamic ioremap arena (K1-BAR); ECAM at the q35 MMCONFIG 0xb0000000; IOAPIC/LAPIC map above the 2 GB bootstrap coverage so walks allocate fresh PMD tables rather than demoting bootstrap leaves.
Β§4aarch64: firmware-less ELF + embedded DTB
- Boot form: QEMU
-kernelwith a plain ELF follows no image protocol β no DTB pointer in x0, and -append/-dtb are ignored; the kernel embeds a full QEMU-virt DTB blob in .rodata (symbol aarch64_virt_dtb; build.rs dumps it fromqemu-system-aarch64 -M virt,gic-version=3 -cpu cortex-a72when the .dtb file is absent); the cmdline comes from the blob's /chosen/bootargs. Every boot requires -M virt,gic-version=3 (the default virt is GICv2; both the driver and the embedded DTB are v3). - Entry: EL2 β drop to EL1h (HCR_EL2 = RW only; all interrupt routing stays at EL1; CNTHCTL opens both timers; CPACR_EL1 FPEN set β NEON traps otherwise).
- Identity-window lifecycle: the TTBR0 identity window covers physical [0,2GB) and is torn down after SMP start β cpu_started(n) β core n needs no low addresses (the secondary stub installs the high-VA VBAR BEFORE reporting itself started; that ordering legitimizes the teardown); the boot core's TTBR0 parks on the kernel root plus a broadcast vmalle1is clears cached translations; low-address usercopies fault into the exception-table fixup β βEFAULT.
- Memory map: linear map phys P β PAGE_OFFSET + P; the kernel image has a 32 MB window; the kernel-side MMIO alias sits at VMALLOC_START (the low window covers physical [0,1GB)); a second window maps the PCI ECAM (physical 0x40_1000_0000, 256 MB, fixed across RAM sizes).
Β§5Kernel image layout and static reserves
- Link-time ASSERTs guard the windows: riscv64
ASSERT(_end <= 0xffffffff81c00000, ...)β the window grew through LAYOUT-2..6 (12β14β18β20β24β26β28 MB; the D-3 symbol table's +3.1 MB once required moving the window, heap base and RAM reservation together); x86_64ASSERT(__kernel_lma_end <= 0x28000000, ...)(40 MB) β historically x86_64 had no guard, and an image outgrowing the window silently corrupted boots (the unreserved live .bss tail). - memblock reserves track the real image end (2df7b612): the static 30 MB reserve once trailed the 31.4 MB SMP8 image by 368 KB β the buddy handed out live .bss, kernel_layout() read garbage, every create_user_address_space root page was rejected, and exec /init failed ENOMEM (misread upstream as "ramfs ELF phdr corruption"; the file bytes were intact). Fix: reserve max(KERNEL_RESERVE_SIZE, kernel_image_phys_end()).
- Numeric incidents: a one-digit KERNEL_RESERVE_SIZE 640 MBβ40 MB typo (the fake-OOM engine β 351 MBβ951 MB of false seeding, 16/16 boots zero OOM); the riscv64 heap base once sat 2 MB inside the image window (the image .bss including the boot stack grew past the heap base; unit-test boots panicked). The rust_main_tail layout guard halts when the image overlaps the heap base.
- Kernel.toml sizing policy: kernel_heap_size = 1024 MB (the OH context: a 512 MB window once sat at ~513/524 MiB, the clone heap-reserve gate <8 MiB tripped, every pthread_create returned EAGAIN, and FFRT services aborted).
Β§6Trap/syscall entry and uaccess
- riscv64: PtRegs 288 bytes #[repr(C)] (a separate orig_a0 slot for syscall rollback); the sscratch/tp swap protocol discriminates the trap source (kernel = sscratch 0; user = sscratch holds the task pointer) with an early-boot fallback; the kernel trap path: per-CPU interrupt-stack selection (via ti_cpu; skipped when nested) β a 288-byte PtRegs β save 31 GPRs β CSRs read after registers β acquire the kernel big lock amoswap.d.aq β clear sscratch. The user-trap exit: save ti_kernel_sp β the signal/resched loop β restore sstatus/sepc/GPRs β release the lock amoswap.d.rl β sc.d clears the LR/SC reservation β restore sscratch β fence β sret.
- x86_64: STAR = (0x23<<48)|(0x08<<32) β SYSCALL enters CS=0x08/SS=0x10 and SYSRET64 returns CS=0x33/SS=0x2b; FMASK = TF|IF|DF β syscalls enter with IF clear, and handle_syscall re-enables interrupts immediately (Linux's own discipline: running a syscall at IF=0 starves ticks/softlockup/preemption; the user rflags are saved untouched so iretq restores user IF); the fast path SYSRETs (with the rcx/r11 mirror checks), everything else IRETQs; fixup_bad_iret matches Linux (the frame owner dies, the system lives); the #UD handler routes user #UD through force_sig_fault(SIGILL) β the OH processdump/CFI reporter runs.
- aarch64: SVC leaves ELR at the instruction AFTER svc β the syscall path never rewinds the PC; ERESTARTSYS rewinds pc β= 4 onto the svc; syscall number x8, args x0-x5, return x0; the pt_regs.syscallno slot plus orig_x0.
- Exception tables: riscv64 records (insn, fixup) pairs in
.ex_table; a uaccess fault looks up the fixup and jumps there, returning the uncopied byte count; x86_64 has the same fixup_exception shape. uaccess: riscv64 brackets copies with the SUM bit (sstatus 18) plus a three-phase asm copy (align / 8Γ-unrolled word copy, WORD_COPY_MIN = 71 bytes); x86_64 uses uaccess_safe.rs + AC brackets (D-18: a pushfq/orq/popfq sequence replaces STAC/CLAC β the STAC/CLAC encodings take an unconditional #UD under QEMU TCG, proven by bisection; the bracket costs ~55 ns and boots on both hardware and TCG; the SMAP-VIOL beacon verifies silence every boot); the PF-NOCTX beacon: SMAP on, AC clear, the fault address in user range and a no_context fault = a raw user-pointer dereference β naming rip/addr before the oops machinery mangles context (capped at 8 prints; "the audit loop is: run the tests, read this line, convert the site to uaccess, repeat").
Β§7The device MMIO alias arena (D-17, x86_64)
- Mechanism: device MMIO is reached exclusively through kernel-side alias windows (two static windows + a dynamic ioremap arena, IOREMAP_ARENA 256 GB); the bootstrap identity map is UNLINKED after the boot-stack switch β no low identity window exists at runtime; firmware-placed low MMIO (the RTC page at 0x101000 etc.) also maps into the arena.
- ioremap_bar_kernel: checks the fixed window first (never burning arena space); a re-registration check (two capabilities sharing one BAR); fresh chunks are 2 MB-aligned and advance DYN_BAR_CURSOR atomically; the DYN_BARS registry (when full, the BAR stays on the identity fallback and warns "user-CR3 MMIO on it will KERNPANIC"); the USER_MMS_EXIST guard warns when mapping after user mms exist (existing user CR3s will not see the new arena link); the call contract is the single-threaded boot context (PCI probe).
- Advantage: one MMIO discipline under every CR3 β trap-context device access cannot fault under a user page table; the uaccess contract has no identity hole (a copy_from_user at low user addresses faults instead of reading physical memory β the prlimit64 EFAULT gap this closed). Compatibility: the x86_64 suite 1194/1194 (including the previously-red EFAULT row); guards = the PF-NOCTX and SMAP-VIOL beacons silent.
Β§8The Kernel.toml configuration system
- build.rs parses Kernel.toml (base) and an optional build/.config overlay β wholesale replacement was once a footgun; the active cargo feature (platform) wins over Kernel.toml's default; the generated config.rs must not be hand-edited.
- The config carries [general]/[platform]/[memory] plus the arch-gated kernel-stack policy (D-2's kernel_stack_size_x86_64); MAX_CPUS reaches trap.S from config (a compile-time clamp).
- print_status boot-table rows: each row is staged into one 80-byte buffer and emitted as ONE batched console write (per-byte putchar once paid a full UART irqsave lock pair plus 16550 per-register traffic Γ ~150 rows).
Canonical docs: docs/architecture/boot.md (the three-port current state), riscv64.md (the riscv deep walk), aarch64.md (the aarch64 canonical map), memory.md (memory init). Code: kernel/src/arch/{riscv64,x86_64,aarch64}/, kernel/build.rs.