Interrupts & timers

kernel/src/interrupt/ + drivers/intc/ + drivers/timer/ + timer/ Β· The IRQ framework, softirqs, the preemption model, IF=0 discipline, the typed timer wheel, NOHZ and the vDSO β€” mechanisms and measured numbers Β· Sources: design-divergences.md (D-19), docs/development/perf-gate-baseline.md (the interrupt guarded domain)

Β§1The interrupt framework

  • The irq_desc model (kernel/src/interrupt/): one descriptor per IRQ holding the action chain; the irq_chip function-pointer table abstracts controller operations; irq_domain performs the hwirqβ†’virq mapping; request_irq/free_irq/request_nmi form the registration surface.
  • Dispatch: riscv64 claims the highest-priority pending IRQ from the PLIC and enters the IRQ framework, with EOI performed by the flow handler via irq_chip.irq_eoi; x86_64 enters trap_handler directly by vector; aarch64 uses the GICv3 claim/EOI pair.

Β§2Vector layout (per architecture)

ArchLayout
x86_64LAPIC timer 0xE0; dedicated IPI vectors 0xE1–0xE5 (CALL_FUNCTION/RESCHEDULE/TLB_FLUSH/STOP/IRQ_WORK β€” one vector per IPI type, skipping the pending-bit handshake); PIC remapped master 0x20-0x27 / slave 0x28-0x2F (IRQ0=32); syscall 0x80; spurious 0xFF. External interrupts stay on the 8259 (virtual wire): LVT LINT0 = ExtINT β€” every EOI must call both the APIC eoi and pic_send_eoi (a leftover in-service state wedges all further interrupt delivery).
riscv64PLIC at 0x0c000000 (accessed through the kernel-side MMIO alias; the identity low window is not cloneable by user MAP_FIXED); two contexts per hart, S-mode = 2Β·hart+1; the priority layering: UART (irq 10) = 5, virtio-mmio (1..=8) = 3, everything else at base β€” pre-fix, every IRQ had priority 1 and the console could not preempt bulk device traffic. The CLINT is M-mode only (OpenSBI v1.3+); S-mode goes through SBI; IPIs use the SBI IPI extension.
aarch64GICv3: INTIDs 0-15 are SGIs (INTID 0 carries the IPI bitmap), 16-31 PPIs (INTID 27 = PPI 11 = the EL1 virtual timer), 32+ SPIs (PL011 = SPI 1); the CPU interface uses the ICC_* system registers; GICD_CTLR under DS=1: bit 1 = Grp1NS enable (writing the secure-world encoding leaves Group-1 forwarding off β€” zero IRQ exceptions, the timer PPI latched in GICR_ISPENDINGR); SGIs go via ICC_SGI1R (TargetList/INTID).

Β§3Preemption model and the IRQ-off discipline

  • preempt_count bit layout: PREEMPT [0:7], SOFTIRQ [8:15], HARDIRQ [16:19], NMI [20:23], PREEMPT_ACTIVE bit 26; the count lives on the Task struct; preemptible() = count==0.
  • irq_enter/exit spans a context switch: HARDIRQ_OFFSET is booked by the task that entered β€” the tick handler may switch inside the bracket, so the per-CPU booked-task stack releases on the booked task; a split is the producer of the "leaked 0x10000 face" (established as the root cause on the 119th epoll-pwait run).
  • Kernel preemption defaults off (explicit schedule() voluntary points); the kpreempt= boot param arms the irq-exit preemption hook; the A/B reads flat for hog regimes and the default stays off.
  • The IF=0 discipline and the official measurement: the EFLAGS-pair probe (syscost v3) reads the real IF flag at every hook and adjudicates the model against it β€” a modeled-open segment found with IF=1 is a leak, closed into the HEAL counters; the HARDIRQ/SOFTIRQ/UART zones split separately; REST call sites attribute via #[track_caller] into a 96-slot table. Official share: IF=0 = 20.4 Β±3pp (pa115debt1; the historical anchor pa27 = 25.7%, the drop's mechanism being phantom-bits moving softirq dispatch out of the irq_exit IF=0 window into schedulable ksoftirqd); the longest closed cli segment in the REST zone is 6–9 ms (band ≀10 ms; the former 594–599 ms one-shots are gone); the 210 ms hardirq-frame one-shot is a named watch row (the starvation bound of inline dispatch).

Β§4The softirq model

  • 10 types: Hi/Timer/NetTx/NetRx/Block/IrqPoll/Tasklet/Sched/Hrtimer/Rcu; a per-CPU pending bitmap plus per-CPU/per-vector raise counters (feeding /proc/softirqs).
  • Dispatch: raised from hardirq; processed in __do_softirq() at irq_exit, overflowing into ksoftirqd. A double budget bound: MAX_SOFTIRQ_RESTART = 10 rounds + 2 ms of wall clock (the Linux MAX_SOFTIRQ_TIME shape) β€” rationale: an unbounded irq-exit budget once let a timer/IO storm pin both CPUs in IRQ context (93% si_busy) with zero scheduling picks.
  • ksoftirqd: one per CPU; schedules on need_resched inside the drain loop (the phantom-bits fairness rule: otherwise a feedback-heavy backlog lets the kthread monopolize its CPU through thousands of 2 ms windows); the re-check-before-schedule lost-wake pattern (the run_ksoftirqd shape).
  • Window progress records: SOFTIRQ_WIN packed {open|handler bit|restart_count|pending} + SOFTIRQ_WIN_T0, cross-CPU readable β€” external forensics for wedged windows; a softirq handler booking SOFTIRQ_OFFSET plus a heal landing mid-window wedges the machine (VW-WEDGE established as the root cause, 29 completions lost).

Β§5The timer subsystem and the typed wheel (D-19)

  • Time base: nominal 10 MHz (100 ns/cycle); the x86_64 timekeeper is the TSC via multiply-shift with HPET calibration (a 1 Hz resync gated), all state in the vDSO data page; the riscv64 time CSR is naturally 10 MHz; aarch64 uses the EL1 virtual timer (CNTV_CVAL).
  • Wheel shape: 64 buckets per level, three levels (granularity 1/64/4096 jiffies, total span 262144 jiffies β‰ˆ 43.7 min at HZ=100) plus a FAR list for deeper horizons (rescanned every 4096 jiffies β‰ˆ 41 s); per-pass work is O(due entries); plain fixed .bss storage with the single TIMER_TABLE lock; delivery outside the lock (R12-3).
  • Six historical bug classes eliminated by the type system: mixed clock-domain comparison (Deadline<JiffyClock>/<HresClock> mixing = compile error), one-shot action leak (the Action enum is exclusive: Wake/Signal/TimerFd), periodicity misuse (Option<NonZeroU64>), the silent if/else-if delivery chain, slot-lifecycle misuse (FiredToken by value), and delivery-context overreach (a SoftirqCtx ZST pins delivery to softirq context). On the real tree: six misuses were all rejected by rustc (E0308/E0599/E0061/E0382).
  • Caller-side guards: WakeArmed owns one registration id (not Copy/Clone β€” a double cancel is unrepresentable); cancel(self) consumes into the WakeSpent tombstone; Drop cancels an unconsumed guard (the pool-exhaustion wedge class); hres arms bake in the hart re-program (an hres sleep whose caller forgets the re-program waits for the next 10 ms grid tick).
  • Same-test platform A/B (s5-boot, OH image, KVM q35 smp2/4G, syscost=1, 780 s; A = the old flat-table kernel, B = the typed wheel): timer arm lock hold 924β†’697 ns, pending 1288β†’868 ns, del 1291β†’862 ns (volume matched within 0.1%); nanosleep average 772.6β†’662.7 ms; pbench p2_ctxsw 12.198 vs 15.400 Β΅s. Cost: slot 56β†’64 B (a 66 KB .bss pool).
  • Superseded shape: a flat 1024-slot array whose expiry was re-checked on every softirq pass, with timer classes encoded as runtime flag combinations.

Β§6NOHZ / tick-sched

  • A function-for-function mirror of Linux's tick-sched.c (check_tick_dependency, tick_nohz_stop/restart_sched_tick, tick_nohz_irq_exit, the nohz_full= parameter); per-CPU TickSched{inidle, tick_stopped, tick_dep_mask, next_tick}; tick_dep bits POSIX_TIMER/PERF_EVENTS/SCHED/CLOCK_UNSTABLE/RCU/RCU_EXP.
  • Structural mappings (recorded in the rust-native register): per-rq β†’ GRQ (the SCHED dependency is evaluated on the fly by can_stop_full_tick; the 0β†’1 kick kept as a resched IPI from the enqueue edge); clockevent KTIME_MAX arm β†’ timer_arm_oneshot(u64::MAX) (the u32 count clamp parks the LAPIC at max delta); tick_do_timer_cpu β†’ the boot CPU with sleep capped at TIMEKEEPING_MAX_DEFERMENT (10 s); perf: the dep tracks any armed sampling event β€” Rux software sampling rides the tick, recorded as a divergence (Linux's per-event hrtimer swevent is the follow-up).
  • Wired on x86_64; the other architectures keep their grid-glue set_next_trigger. Non-nohz_full CPUs keep the byte-identical grid path.
  • Measured (4+4 interleaved lmbench lat_syscall,null runs, 6 hostile external QEMUs): the main tip passes 0/10; nohz_full=1 β†’ 11/12 (92%); p1_null median 0.050 vs 0.060 Β΅s (βˆ’17%); both census bands hold exactly; GRQ average hold 699 ns (a release-class cross-class reading, not comparable to the debug band); a pre-nohz parent control confirmed the p2 drift belongs to board spread, not the nohz change.

Β§7The vDSO

  • Structure: every execve installs a two-page user mapping β€” a data page (RW, the kernel-updated time base) and a code page (RX, a minimal ELF with three symbols); AT_SYSINFO_EHDR points at the code page; glibc/musl call __vdso_clock_gettime/gettimeofday/clock_getres directly.
  • Clock set: REALTIME(0)/MONOTONIC(1)/MONOTONIC_RAW(4)/BOOTTIME(7)/TAI(11 = REALTIME + 37); on x86_64 any other clockid falls back inside the vDSO to the real syscall.
  • Bit-equivalence design (x86_64, the unified track): read_time()'s TSC branch and the vDSO functions compute the same value from the same words with the same integer operations β€” nom(t) = base_mono_nom + ((tsc(t) βˆ’ cycle_last) Γ— mult) >> 32, integer-associative so the two tracks are bit-identical (what LTP clock_gettime04 demands); base_mono_nom is nominal-domain and monotone; the seq word doubles as the writer lock (cmpxchg evenβ†’odd), closing the multi-writer publication tear; the data page is user-RO β€” precisely because the kernel trusts its words.
  • The rejected fast path (on record): per-CPU correction publication carried the TSC_CPU_OFF spread β€” tsleep2 βˆ’356 ms at t<1 s; the syscall-vs-vdso spread to +1.2 ms; a clock_gettime04 residual of βˆ’4582 ns. The unified track replaces the correction chain with bit-equivalence.
  • The riscv64 lesson: the seq word must be read THROUGH the page pointer β€” a former separate VDSO_SEQ static was never aliased to the page, so a reader descheduled between its base_tsc and base_mono_ns loads paired a stale tsc with a fresh mono (a vDSO clock ahead of the syscall clock by the whole preemption gap; LTP clock_gettime04 "Time travelled backwards βˆ’454618735 ns").
  • Measured (name-only same-test platform A/B, tok-r4fix vehicle, KVM q35 SMP2 4G, 780 s): clock_gettime syscalls per OH boot 8,259,773 β†’ 0 (βˆ’29.4 CPU-s = βˆ’2.0% of the machine, βˆ’26% of total syscall volume), the nanosleep average and the GRQ lock-site average flat, zero panics on both sides; LTP clock_gettime04 6/6 including the vdso variants and both COARSE clocks.

Β§8IPI costs and TLB shootdowns

  • The resched IPI sender side alone is 935–1139 ns; IPI+halt exit ~2–5 Β΅s of KVM exit; wakes inside the 120 Β΅s polling idle window elide the IPI (Dekker SeqCst pairing for lost-wake safety).
  • The TLB-shootdown lazy mm_cpumask: pre-fix, every munmap/fork-COW/mprotect/COW fault broadcast a full-flush IPI to all CPUs β€” a pristine OH boot counted 4,031,356 by t=200s (~20k/s); the per-mm cpu_mask (set at switch-in, cleared at switch-away, grow-only under PCID) reduces this to 10,209 (βˆ’99.7%) with flush semantics unchanged. Mechanism details on the memory-management Β§10.
Code: kernel/src/interrupt/ (preempt/softirq/ksoftirqd/irqchip), kernel/src/drivers/intc/ (apic/plic/clint/gicv3), kernel/src/drivers/timer/, kernel/src/timer/ (wheel/oneshot/nohz), kernel/src/mm/vdso.rs. Divergence: D-19 (the typed wheel). Gate bands (the interrupt guarded domain): IF=0 20.4Β±3pp, max cli REST ≀10 ms, SOFTIRQ 3.1Β±2pp. Instruments: the syscost IF=0 EFLAGS-pair probe, tick-mode shares, rux:schedtrace (see performance debugging).