Boot flow

kernel/src/arch/*/ + main.rs Β· The three boot protocols, early memory, kernel image layout, SMP bring-up, trap/syscall entry and the device-address discipline Β· Sources: docs/architecture/boot.md, riscv64.md, aarch64.md, design-divergences.md (D-17/D-18)

Β§1Boot invariants

All three architectures share an explicitly recorded set of boot invariants (boot.md):

  • Pre-allocator code uses static BSS pools only β€” every early-stage data structure is self-sufficient without a heap;
  • The boot CPU owns all early initialization; secondaries spin on a release flag until the scheduler exists;
  • The init= resolution and the PID-1 contract sit at the end of rust_main β€” the ltpscan and lmbench runners are themselves PID-1 ELFs.

Generic sequence (all three ports): early arch bring-up (page tables, serial) β†’ memory init (memblock, zones, vmemmap, the small-object allocator) β†’ traps/IRQs β†’ scheduler and init task β†’ driver probe β†’ mount root β†’ run init.

§2riscv64: OpenSBI M→S

  • Boot chain: QEMU loads OpenSBI (fw_dynamic) in M-mode and hands the S-mode kernel the entry at _start (a0 = hartid, a1 = DTB pointer); boot.S sets the stack, clears BSS PC-relative, builds the trampoline page table (first 2 MB kernel VAβ†’PA) then the full early page table, enables the MMU and jumps to VA to enter rust_main.
  • SMP bring-up: OpenSBI HSM starts all harts; the first to reach init() via CAS becomes the boot hart; the rest wait on SMP_INIT_DONE with wfi.
  • The R87 secondary bring-up window (lesson): secondaries once waited with the timer already enabled + wfi β€” during the whole BSP tail (fs mounts, drivers, init spawn; scales with -m), every tick could exit through irq_exit preemption into __schedule() while PER_CPU[hart] was still a zeroed slot; the zero slot triggered heal chains, the hart executed enqueued tasks while holding its boot stack, and slot fights booked the wrong prev at a context switch β€” mixed-context save (one task's registers written into another's thread record), detonating as the "kill the idle task" KERNPANIC. Fix: spin (not wfi β€” QEMU's wfi only wakes on an ENABLED pending interrupt) until is_boot_complete, and enable timer/external/software interrupts only AFTER init_secondary publishes PER_CPU[hart]. Companion finding: boot.S zeroes sie on every hart while ipi::init only sets SSIE on the boot hart β€” without per-hart SSIE, a secondary ran with all IPIs masked (resched/TLB/call-function pending forever; tick-quantized futex wakeβ†’run 9118 Β΅s vs 24 Β΅s same-hart). The rule on record: a secondary hart takes no interrupts until its per-CPU state is fully initialized and it has entered the idle loop (Linux's own rule).
  • Per-CPU interrupt stacks: 16 KB per CPU, 16-byte aligned; used for kernel-mode traps (user traps use task kernel stacks).

Β§3x86_64: multiboot1 + bzImage dual protocol

  • The multiboot1 path: QEMU -kernel ELF, entered in 32-bit protected mode (eax = 0x2BADB002, ebx = the multiboot info pointer); boot.S saves the pointer, builds the early page tables (identity low + higher-half kernel map) and enables long mode; the multiboot blob lives in low memory β€” everything needed is copied into kernel BSS immediately (memory map, cmdline, bootloader name), because the bootstrap identity mapping dies once the real page tables install.
  • The bzImage path (Linux boot-protocol header + 16-bit trampoline): 16-bit β†’ protected mode (hardcoded absolute far jumps) β†’ 0x100000 (the 32-bit boot stub) β†’ long mode β†’ kernel; the same image boots where a Linux-boot-protocol loader is expected.
  • Lifecycle of the bootstrap identity map: the bootstrap PML4 carries the low identity window (serving only the boot stack); after mm::init() builds the real higher-half tables and switches CR3, PML4[0] is UNLINKED immediately after the stack switch (drop_bootstrap_identity) β€” no low identity window exists at runtime (D-17, Β§7).
  • Above-4G BARs: the host bridge may place virtio 64-bit BARs at arbitrary above-4G addresses β€” two static alias windows plus a dynamic ioremap arena (K1-BAR); ECAM at the q35 MMCONFIG 0xb0000000; IOAPIC/LAPIC map above the 2 GB bootstrap coverage so walks allocate fresh PMD tables rather than demoting bootstrap leaves.

Β§4aarch64: firmware-less ELF + embedded DTB

  • Boot form: QEMU -kernel with a plain ELF follows no image protocol β€” no DTB pointer in x0, and -append/-dtb are ignored; the kernel embeds a full QEMU-virt DTB blob in .rodata (symbol aarch64_virt_dtb; build.rs dumps it from qemu-system-aarch64 -M virt,gic-version=3 -cpu cortex-a72 when the .dtb file is absent); the cmdline comes from the blob's /chosen/bootargs. Every boot requires -M virt,gic-version=3 (the default virt is GICv2; both the driver and the embedded DTB are v3).
  • Entry: EL2 β†’ drop to EL1h (HCR_EL2 = RW only; all interrupt routing stays at EL1; CNTHCTL opens both timers; CPACR_EL1 FPEN set β€” NEON traps otherwise).
  • Identity-window lifecycle: the TTBR0 identity window covers physical [0,2GB) and is torn down after SMP start β€” cpu_started(n) β‡’ core n needs no low addresses (the secondary stub installs the high-VA VBAR BEFORE reporting itself started; that ordering legitimizes the teardown); the boot core's TTBR0 parks on the kernel root plus a broadcast vmalle1is clears cached translations; low-address usercopies fault into the exception-table fixup β†’ βˆ’EFAULT.
  • Memory map: linear map phys P β†’ PAGE_OFFSET + P; the kernel image has a 32 MB window; the kernel-side MMIO alias sits at VMALLOC_START (the low window covers physical [0,1GB)); a second window maps the PCI ECAM (physical 0x40_1000_0000, 256 MB, fixed across RAM sizes).

Β§5Kernel image layout and static reserves

  • Link-time ASSERTs guard the windows: riscv64 ASSERT(_end <= 0xffffffff81c00000, ...) β€” the window grew through LAYOUT-2..6 (12β†’14β†’18β†’20β†’24β†’26β†’28 MB; the D-3 symbol table's +3.1 MB once required moving the window, heap base and RAM reservation together); x86_64 ASSERT(__kernel_lma_end <= 0x28000000, ...) (40 MB) β€” historically x86_64 had no guard, and an image outgrowing the window silently corrupted boots (the unreserved live .bss tail).
  • memblock reserves track the real image end (2df7b612): the static 30 MB reserve once trailed the 31.4 MB SMP8 image by 368 KB β€” the buddy handed out live .bss, kernel_layout() read garbage, every create_user_address_space root page was rejected, and exec /init failed ENOMEM (misread upstream as "ramfs ELF phdr corruption"; the file bytes were intact). Fix: reserve max(KERNEL_RESERVE_SIZE, kernel_image_phys_end()).
  • Numeric incidents: a one-digit KERNEL_RESERVE_SIZE 640 MBβ†’40 MB typo (the fake-OOM engine β€” 351 MBβ†’951 MB of false seeding, 16/16 boots zero OOM); the riscv64 heap base once sat 2 MB inside the image window (the image .bss including the boot stack grew past the heap base; unit-test boots panicked). The rust_main_tail layout guard halts when the image overlaps the heap base.
  • Kernel.toml sizing policy: kernel_heap_size = 1024 MB (the OH context: a 512 MB window once sat at ~513/524 MiB, the clone heap-reserve gate <8 MiB tripped, every pthread_create returned EAGAIN, and FFRT services aborted).

Β§6Trap/syscall entry and uaccess

  • riscv64: PtRegs 288 bytes #[repr(C)] (a separate orig_a0 slot for syscall rollback); the sscratch/tp swap protocol discriminates the trap source (kernel = sscratch 0; user = sscratch holds the task pointer) with an early-boot fallback; the kernel trap path: per-CPU interrupt-stack selection (via ti_cpu; skipped when nested) β†’ a 288-byte PtRegs β†’ save 31 GPRs β†’ CSRs read after registers β†’ acquire the kernel big lock amoswap.d.aq β†’ clear sscratch. The user-trap exit: save ti_kernel_sp β†’ the signal/resched loop β†’ restore sstatus/sepc/GPRs β†’ release the lock amoswap.d.rl β†’ sc.d clears the LR/SC reservation β†’ restore sscratch β†’ fence β†’ sret.
  • x86_64: STAR = (0x23<<48)|(0x08<<32) β€” SYSCALL enters CS=0x08/SS=0x10 and SYSRET64 returns CS=0x33/SS=0x2b; FMASK = TF|IF|DF β€” syscalls enter with IF clear, and handle_syscall re-enables interrupts immediately (Linux's own discipline: running a syscall at IF=0 starves ticks/softlockup/preemption; the user rflags are saved untouched so iretq restores user IF); the fast path SYSRETs (with the rcx/r11 mirror checks), everything else IRETQs; fixup_bad_iret matches Linux (the frame owner dies, the system lives); the #UD handler routes user #UD through force_sig_fault(SIGILL) β€” the OH processdump/CFI reporter runs.
  • aarch64: SVC leaves ELR at the instruction AFTER svc β€” the syscall path never rewinds the PC; ERESTARTSYS rewinds pc βˆ’= 4 onto the svc; syscall number x8, args x0-x5, return x0; the pt_regs.syscallno slot plus orig_x0.
  • Exception tables: riscv64 records (insn, fixup) pairs in .ex_table; a uaccess fault looks up the fixup and jumps there, returning the uncopied byte count; x86_64 has the same fixup_exception shape. uaccess: riscv64 brackets copies with the SUM bit (sstatus 18) plus a three-phase asm copy (align / 8Γ—-unrolled word copy, WORD_COPY_MIN = 71 bytes); x86_64 uses uaccess_safe.rs + AC brackets (D-18: a pushfq/orq/popfq sequence replaces STAC/CLAC β€” the STAC/CLAC encodings take an unconditional #UD under QEMU TCG, proven by bisection; the bracket costs ~55 ns and boots on both hardware and TCG; the SMAP-VIOL beacon verifies silence every boot); the PF-NOCTX beacon: SMAP on, AC clear, the fault address in user range and a no_context fault = a raw user-pointer dereference β€” naming rip/addr before the oops machinery mangles context (capped at 8 prints; "the audit loop is: run the tests, read this line, convert the site to uaccess, repeat").

Β§7The device MMIO alias arena (D-17, x86_64)

  • Mechanism: device MMIO is reached exclusively through kernel-side alias windows (two static windows + a dynamic ioremap arena, IOREMAP_ARENA 256 GB); the bootstrap identity map is UNLINKED after the boot-stack switch β€” no low identity window exists at runtime; firmware-placed low MMIO (the RTC page at 0x101000 etc.) also maps into the arena.
  • ioremap_bar_kernel: checks the fixed window first (never burning arena space); a re-registration check (two capabilities sharing one BAR); fresh chunks are 2 MB-aligned and advance DYN_BAR_CURSOR atomically; the DYN_BARS registry (when full, the BAR stays on the identity fallback and warns "user-CR3 MMIO on it will KERNPANIC"); the USER_MMS_EXIST guard warns when mapping after user mms exist (existing user CR3s will not see the new arena link); the call contract is the single-threaded boot context (PCI probe).
  • Advantage: one MMIO discipline under every CR3 β€” trap-context device access cannot fault under a user page table; the uaccess contract has no identity hole (a copy_from_user at low user addresses faults instead of reading physical memory β€” the prlimit64 EFAULT gap this closed). Compatibility: the x86_64 suite 1194/1194 (including the previously-red EFAULT row); guards = the PF-NOCTX and SMAP-VIOL beacons silent.

Β§8The Kernel.toml configuration system

  • build.rs parses Kernel.toml (base) and an optional build/.config overlay β€” wholesale replacement was once a footgun; the active cargo feature (platform) wins over Kernel.toml's default; the generated config.rs must not be hand-edited.
  • The config carries [general]/[platform]/[memory] plus the arch-gated kernel-stack policy (D-2's kernel_stack_size_x86_64); MAX_CPUS reaches trap.S from config (a compile-time clamp).
  • print_status boot-table rows: each row is staged into one 80-byte buffer and emitted as ONE batched console write (per-byte putchar once paid a full UART irqsave lock pair plus 16550 per-register traffic Γ— ~150 rows).
Canonical docs: docs/architecture/boot.md (the three-port current state), riscv64.md (the riscv deep walk), aarch64.md (the aarch64 canonical map), memory.md (memory init). Code: kernel/src/arch/{riscv64,x86_64,aarch64}/, kernel/build.rs.