fio 3.36

Storage-I/O performance comparison (same image, same binary, same test platform): throughput and latency; an engine row without numbers is unavailable · test platform and method

§1Data (same image, same binary, same test platform)

Both sides run the same ext4 image, the same static fio 3.36 binary, the same job file — Rux (main release) and Linux (same-image guest kernel 7.0.0-38) in the same test platform (q35/KVM/-cpu host/SMP8/8G, virtio-blk; interleaved boots ×3, medians); direct=1 bypasses the page cache. Each row carries its newest measured value; same-window kernel-vs-kernel ratios drift with host-load windows on BOTH sides (±10% class) — ratios are meaningful only within one window. An engine row without numbers is currently unavailable.

JobRuxLinux (same test platform)Ratio (Rux/Linux)
4k random write 8M + fsync=1 (durability path, 15 s)20.1–22k IOPS15.3–16.2k IOPS (fsync median ~57 µs)Rux ~1.24× ahead
4k random rw 40M (direct, 50/50)22.15k/22.15k IOPS22.0k/22.0k IOPS0.96× (parity band)
4k random write 40M (direct)36.75k IOPS~37.9k IOPS0.97×
4k random read 40M (direct, 20 s)37.0k IOPS (numjobs=1) · numjobs=8 0.91×~41k IOPS0.90× (numjobs=1)
1M sequential write 48M (direct, allocated territory)4000 MiB/s5.2 GiB/s · clat ~188 µs0.83×
1M sequential write 48M (direct, fresh file)4000–4800 MiB/s (median 4000)5.2 GiB/s (same value both shapes)0.75–0.9× (matches overwrite)
1M sequential read 48M (direct)8000 MiB/s · requests/op 4→19.4 GiB/s · clat 90 µs0.83× (merge + SG chains)
1M buffered write 48M (page-cache basis, both sides)11.7 GiB/s · clat 78.4 µs9.4 GiB/s · clat 89.5 µsRux 1.19–1.25× ahead
128MB flush-wall probe (mmap full-sync writeback, total wall clock)pending re-measure1.223 s (old basis)—
libaio (io_setup/io_submit)—yes—
io_uring (the 5.1 ABI enter contract)—yes—

Notes: the fresh-file sequential-write row now matches the overwrite row (async unwritten-extent conversion + delalloc reservation landed). Buffered write: the shared cold-frame RFO rules out the 2× target; the former ~6% gap has been closed by region-based batch allocation (alloc term 55k→14k cycles/op — four shared-line locked writes per page became once per batch); now 1.19–1.25× ahead. The sequential-read gap was closed by request merging + SG chains. The flush-wall probe awaits the next SMP8 re-measure (the fsync path was rebuilt). The 4k random-read row's 0.90× is the numjobs=1 basis; numjobs=8 reads 0.91×. Rows without data: libaio is an unimplemented UAPI (io_setup ENOSYS); io_uring's enter contract triggers a user-space crash (ring semantics incomplete). The posixaio engine runs; its throughput readings join the extension queue. The 48M job-size cap comes from the image's free space — the same constraint on both sides.

§2Optimization method

Gaps are attributed by a two-sided same-test platform accounting (both kernels on the same image and workload, per-op cost decomposed segment by segment via a shared tracefs event timeline + phase cycle brackets + perf on both sides: syscall entry / block resolution / submit+kick / device / completion IRQ / wake / return), then each is rebuilt along Linux's implementation shape: the direct path submits physically consecutive runs in one request; MSI-X with completions walked inside the hardirq vector; synchronous waits poll briefly before sleeping (the completion object is a plain kernel value — at depth 1 the whole sleep machinery costs more than the device round trip; Linux makes the same trade, only opt-in); bulk unwritten-extent conversion with neighbor merge (one tree edit per write span); per-inode dirty-block notes → an O(dirty) fsync walk (the tagged-page-tree analog) with the inode-table block riding the same flush window; buffered writes copy user pages straight into page-cache frames (the copy_page_from_iter shape); directory reads fill incrementally; path and descriptor costs (conditional-refcount close = the fget_rcu shape, readahead state embedded in File); empty leaf tables freed on address-space teardown. Every change is A/B'd on the same test platform, image and binary; negative results stay on record (MSI destination steering is a dead end under this QEMU/KVM+irqfd). Method and per-item evidence: docs/test/perf-campaign-2026-11-17.md.

§3Test platform and method

  • The pair: Rux (main release) vs Linux (same-image guest kernel 7.0.0-38), the same QEMU q35 + KVM (-cpu host SMP8/8G), the same ext4 image (reflink copy, booted snapshot=on), the same static binary (source tarball committed; built by userspace/fio/build-x86.sh); interleaved boots ×3, medians.
  • Protocol: one boot per side runs all jobs in sequence (the qsrun cmds protocol); fio terse single-line output goes into the serial capture with the bw/IOPS/latency fields.
  • Basis: direct=1 rows are the real device path; the buffered-write row is the page-cache-absorbed basis (both sides); ratios are meaningful only within one session.

§3Extension

# add jobs: append .cmds under userspace/ci/classes/ (line format: <budget-s> fio --name=... --filename=/test/qa/...)
# the pair run: run/bench-pair.sh (the Rux side + the Linux side, same image and cmds)
Latency percentiles (dumped JSON) hit the qsrun tail truncation; the terse single line is the current capture basis. Per-job latency grading is in the queue.