Performance

Verdict first, data right after: vs Linux on the same host and the same accelerator (QEMU q35 + KVM) · data as of 2026-11-16 · baseline file docs/development/perf-gate-baseline.md

§1Verdict: the lead/lag picture vs Linux

Every comparison against Linux on this page is same-host, same-accelerator: the same host (9950X), the same QEMU q35 + KVM (-accel kvm -cpu host), the same image shape, the same static benchmark binaries. No native columns.

📏lmbench same-test platform: 14 of 21 cases ahead or at parity, 7 behind

Ahead: lat_pipe 4.6×, lat_ctx 2P 4.0–4.5×, lat_ctx 8P 3.6×, bw_pipe 3.7×, lat_sem 2.9×, bw_unix 2.1×, lat_unix 1.4×, fstat 1.3×; bcopy/bzero/mem_rd/usleep at or above parity, bw_file_rd faster than the timer granularity.

Behind (Rux/Linux): open 1.92×, read_cached 1.84×, stat 1.28×, write 1.06×; null now at parity (0.97×); fork/exec 0.75–0.79× (ahead). Storage and real-time rows below.

best lead 4.6× · worst lag 2.5× · data on the lmbench page

⚡Storage and real-time rows: phase-probe attribution, rebuilds along Linux's shapes

Storage I/O is attributed by a two-sided same-test platform accounting (shared tracefs timeline + phase brackets + perf on both sides), then rebuilt along Linux's shapes: the fsync path rebuilt (bulk unwritten-extent conversion with neighbor merge, per-inode dirty-block notes with an O(dirty) walk, the inode-table block riding the same flush window) — 0.30× to ~1.24× AHEAD; synchronous waits poll briefly before sleeping (4k randwrite 0.97×, randrw 0.96×, randread 0.90× — the parity band); the direct path submits physically consecutive runs in one request (1M sequential read 226× behind → 0.62×); buffered writes copy user pages straight into frames (0.94×, the test platform's physical floor: the cold-frame RFO is paid by both sides); directory reads fill incrementally (getdents 1.95× ahead); conditional-refcount close + readahead state embedded in File (openclose 1.59×, read_cached 1.66× — the remaining term is the open-side path walk). Real-time: timer expiry at the top of the hardirq vector (rt_wakesys 3.3 µs, parity), PI futex in Linux's shape (pi_stress 132,074 inversions), cyclictest FIFO avg 104→69 µs (max 263; same-test platform interleaved re-measurement in flight).

Per-item rows: fio and rt-tests · method and evidence: docs/test/perf-campaign-2026-11-17.md

🏁Geekbench 5: single-core 2465 / multi-core 7504

Single-core total 2465 vs 2165 (114%), multi-core total 7504 vs 7027 (107%). Per-workload split from the 2026-11-04 full capture: multi-core 8 ahead / 5 parity / 8 behind; single-core 13 ahead / 3 parity / 5 behind.

per-workload on the Geekbench page · test platform note: Linux side is the host 4-CPU pin (taskset, topology-matched)
Gap structure: the leads cover scheduler/wakeup (context switch, pipe, semaphore), IPC bandwidth (unix, pipe) and memory bandwidth; the remaining gaps concentrate in syscall entry and path resolution (null/read/write/stat/open at 1.2–2.5×). Under the older Linux-side window the historic campaign had scored fork/exec at parity; the 2026-11-16 same-test platform re-measure puts them in the 1.08–1.19× band (the Linux switch family is measurement-window-sensitive — ratios are only meaningful within one session). The 5 correctness bugs the campaign found and fixed are listed after the §2 tables.

§2The suite map (one page per suite)

SuiteTypeStatusPage
lmbench 3.0-a9microbenchmarks (latency + bandwidth vs same-test platform Linux)the 2026-11-16 full standing + three perf-2x roundslmbench
Geekbench 5.5.1the composite suite (21 workloads vs 4-CPU Linux)SC 2465 / MC 7504, bit-identical reproduction confirmedGeekbench 5
fio 3.36storage I/O (engine enumeration + data integrity + group throughput)same-image same-binary same-test platform comparison (throughput/latency/durability path)fio
rt-tests v2.11 (cyclictest/hackbench/pi_stress)real-time latency + scheduling stresscyclictest/hackbench/pi_stress same-test platform comparison (the newest round)rt-tests
internal microbenchmarks (pbench + the Tier bands + the flush wall + cctx)gate collectors and historical A/BsTier 1 bands + the flush-wall same-test platform comparison + the switch anatomyInternal microbenchmarks
planned (no data, nothing listed)netperf/iperf3, bonnie++/postmark/dbench, UnixBench, a kselftest subset, Trinity/syzkaller corporathe qa-suites onboarding sequence—

§3Method and test platform

  • Same host, same accelerator: both sides on the same host, the same QEMU q35 + KVM (-accel kvm -cpu host), the same Ubuntu rootfs shape, the same static benchmark binaries; one case per boot (the serial console interleaves with long sweeps, so a single-case boot is the only clean collection), PID 1 runner, 35 s in-guest budget (exceptions noted per case).
  • Comparable-class rule: debug vs release-strong never cross-compared (strong reads 10–30× lower on the floor families); TCG vs KVM never cross-compared.
  • Anchor provenance: gate anchors read only Rux's own medians — the Linux column never enters a Rux gate anchor (leg 32 found four cells transcribed from the Linux column; corrected and filed).

§4The merge-gate performance dimension (three tiers + six domains)

Every merge is reviewed for both function and performance; baselines move only through merges that passed their own tier; a regression blocks the merge until fixed or justified in writing. Workflow details: Contributing §3.

TierApplies toRequirement
Tier 1All kernel changesOne quiet syscost boot: census in band, ticks ≥99.5/s, zero new panic/OOM classes, syscall volume ±10%, per-op averages ±20% or better
Tier 2Hot paths: mm/ sched/ fs/ ipc/ binder/ timer/ arch trapPlus one pbench boot: P1–P7 medians within ±20% or better
Tier 3Allocator or IRQ pathsPlus an IF=0 measurement (EFLAGS-pair probe if0 v3, official — no proxy)

Notes: census = boot-time scheduling-event counters; per-op averages = read/write/clock_gettime/nanosleep; the current Tier 1 bands are itemized on the internal-microbenchmarks page.

Six guarded domains

DomainGuarded metric (current band)
Memory allocation / reclaimsmall_alloc global class-lock sections ≤25M/boot (measured 19.7M, latest board 6.0M), REST share ≤10% (now 0.1–1.2%), no reclaim slow-path emergence on standard boots
SchedulerGRQ critical section ≈60 ns (production) / 75–80 ns (minimal bracket), IF=0 span avg 2786 ns, census TERM 8±2 / REFUSED 32±4, P2 ±20%, futex wake→run 7.5 µs ±20%
Filesystem hot pathsP6 rows ±20% (stat 71 / openclose 156 / append 77 / mkdrmdir 2760 / unlink 808 / write_fsync 14200 µs), buffer-cache hit rate 99.1% ≥ baseline, no new sync-write classes
Interrupts / IRQ pathsIF=0 share 20.4%±3pp, longest closed cli segment ≤10 ms (now 6–9 ms), SOFTIRQ share 3.1±2pp, no new IRQ-OFF print classes
IPCp5 family ±20% (p5a 18.3 / p5b 23–41 / stream 24–27 µs), unix writev/truesize 24 µs class not regressed
binderSLOW-TXN classes stable (queue ≤10 ms; per-code bands), throughput not regressed, REST watch row (avg 9.2 µs / max 73 ms, not reproduced)

Notes: the scheduler domain's in-lock figure is the exact-probe basis (tree work ~20–30 ns + D-1 bookkeeping); the 70–84 ns lock wait is the KVM cross-vCPU locked-instruction tax (Linux's per-rq lock pays it too); the IF=0 span average is the call-site basis including acquisition wait (n=6.3M); futex wake→run is the quiet-load basis.

Principle: every subsystem that can affect performance must be under gate measurement — if no existing tool measures it, add a debug observability tool (gated off by default). Measurement is part of the change, not an afterthought.

§5Landed performance work (same-test platform A/B measured)

Each item below was attributed with a same-test platform A/B; gains and costs are recorded in the same table (see docs/architecture/rust-native-register.md):

ItemMechanismMeasured result
Context switchFSGSBASE (393 ns/switch) + lazy FPU (xsaveopt moved into #NM) + CPUID-gated PCIDlat_ctx 4.97→2.07 µs (−58%)
Measured-wait wakeup preemption gate (D-14)Poke a busy CPU only when the wakee's measured wait ≥6 ms, replacing vruntime-credit comparison74-byte unix-stream wake/preempt/switch round trip ~275→~2.5 µs; p5 family 10–18×
fs metadata re-anchorD-8 direct-write jbd2 + D-12 read-window batch fill + D-13 batched publication / discard-on-freestat −65% / openclose −70% / append −88% / mkdrmdir −88%; p6_mkdrmdir median 29,796→2,107 µs
Filesystem flush (fs-flush + virtio-async)run-batched flush + batched async span writes (CHUNK=64, absent-durable skip); the fsx revive-in-place fixfsync 11.55→6.18→3.04 s, msync 13.95→3.31 s (4.2×), sync 11.30→3.31 s
vm-op familyleaf-table direct unmap (allocation-free per-page walker)mmap 72→27–32 µs, munmap 177→52 µs; OH workload 15→5 s/CPU-min
Anon reclaim (SP-7)reclaim check-path reworkcheck rate 35%→82–89%, swap writes +75%
Typed timer wheel (D-19)clock-domain typing + scope guardsarm lock 924→697 ns; nanosleep 772.6→662.7 ms; p2_ctxsw 12.198 vs 15.400 µs
dcache lockless reads (WalkClaim)reader-bank publication + deferred retirementstat 0.643→0.423 µs, open 1.128→0.889 µs
small_alloc magazineper-CPU magazine (depth 16) + bitmap routingclass-lock sections 25M→19.7M→6.0M/boot; REST share 9.2%→0.1–1.2%
GRQ hold0.7 ms granularity pins churn + slimmer pick path (segment-measured)critical section ≈60 ns (production) / 75–80 ns (minimal bracket); the 70–84 ns lock-wait segment = the KVM cross-vCPU locked-instruction tax
unix bandwidthunix staging eliminatedbw_unix 2.6→42.9 GB/s
Page-fault path (campaign r6)backward-fault window + contiguous put_range + PCACHE_RO alias protocol + batched pin release + single TLB flush per munmaplat_pagefault 0.468→0.029–0.031 µs (16.1× total), now 2.26–2.45× ahead of Linux
Hook-call tax & path_memo (campaign r5/r7)1,331 cli/sti outlined calls inlined + path_memo/lease gatesnull −7% / read −5% / write −6% / ctxsw −6.1% / pipe −12%; open −15.2% cumulative, stat −4.6%, read flipped ahead
Word-at-a-time memo hash + capability-only DAC fast path (r10)word-at-a-time hashing; DAC checks take the capability bit onlyopen −5.2%, stat −10%
fd-table bitmap sweeps (r11)fork copy and exit drop as bitmap scans; a ptrace tracee-count gatefork copy 910→574 ns, exit drop 4491→825 ns, premm 4635→757 ns; exit body 17.9→7.3 µs; exec three-state anchored at parity
Fork bistability root cause (r12–14)freeze-forensics chain: a batched UART print storm → the stale-rsp0-at-free UAF fixedhangs 3/6→1/6; fork into the 1.02–1.05× parity band; the call_rcu contract and a wake-monitor inversion fixed alongside

Notes: the flush-row figures are the 128 MB mtest06 shape (x86_64 KVM), fsync ~3.8× cumulative vs pre-campaign; mmap2 TIMEOUT→PASS, fsx -N10000 PASS, pbench fs within the allowed band. The path_memo/lease gates = per-inode open_count replacing the global BTreeMap lock, reader-side seqlock lock-free. The File-pool hypothesis was refuted by PC3 phase anatomy (fil=19/ins=24 ns). The stale-rsp0 UAF fix is six-leg gated.

§6Optimization queue (open gaps, priority order)

  • COW-fault cross-CPU shootdown: the +58% (18.42 µs) honest gap at SMP-8, affecting fork-heavy service graphs — the next main target.
  • lat_syscall open 1.61× / stat 1.32×: open's tail (memo-hit 80–90 ns + fd install); stat's tail v3 unblocked now that the call_rcu contract is fixed — retry.
  • lat_syscall null 1.18× / write 1.14×: entry floor; null's 2× is proven physically unreachable (the shared syscall RTT floor) — converge to parity; write is 6 ns from parity.
  • Geekbench single-core residuals: Text Compression 94% / Navigation 81% / PDF 85% / Clang 88% / Face 87% await isolated re-runs (AES-XTS root-caused and fixed, 105–106%; single-core 2465 = 114%).
  • Network family: bw_tcp TIMEOUT / lat_connect 14–16 ms — a scheduler stale-claim refuses the accept server/worker wakes (WAKE-HEAL-SKIP ×39); TCP throughput needs that fix first.
  • Backfill queue: a fio throughput benchmark (same-test platform Linux baseline, direct=1); the Geekbench same-test platform KVM comparison; page-cache hit-rate absolutes, PCP watermarks, binder throughput.

§7debug measurement toolchain

  • syscost: boot census (syscall volume, census counts, tick rate, per-op averages) — the Tier 1 collector.
  • pbench (test/pbench.c): the P1–P7 microbenchmark probes — the Tier 2 collector.
  • census / wakewatch: scheduler-stall census (WAKE-STRANDED) and wake→run latency forensics (1024-slot side tables + pick-skip reason table).
  • if0 v3: EFLAGS-pair probe — the official IF=0 share and cli-segment measurement (on main; no more tick-mode proxy).
  • fsw / mmprof counters: fs sync-write classes, memory accounting reconciliation (OOM forensics).
  • run/perf-gate.sh: gate automation (anchor JSON + tier dispatch + band verdicts).
  • Unified tracepoint framework: 69 events / 17 groups, byte-exact tracefs, +13–19 ns per record when enabled (see Observability).