SPDK (Storage Performance Development Kit)
SPDK is an open-source set of libraries and tools for writing high-performance, scalable, user-mode storage applications. By bypassing the Linux kernel I/O stack through user-space NVMe drivers and polling-based I/O, SPDK achieves sub-10µs latency and tens of millions of IOPS from a single CPU core.
How SPDK Works
Traditional Linux storage I/O goes through multiple kernel layers (VFS → block layer → device driver → NVMe HW). Each layer adds context switches, memory copies, and interrupt overhead. SPDK eliminates this by:
- User-space NVMe driver — SPDK maps NVMe device registers directly into user-space memory via VFIO, bypassing the kernel entirely.
- Polling instead of interrupts — A dedicated CPU core spins continuously checking for I/O completions. This trades CPU cycles for consistent sub-microsecond latency.
- Lock-free, zero-copy I/O — Data buffers are pinned in huge pages and passed directly to the NVMe device DMA engine.
SPDK and NVMe-oF
SPDK is a common implementation foundation for high-performance NVMe-oF targets. The SPDK NVMe-oF target can serve millions of IOPS per server node with single-digit microsecond latency. Projects like simplyblock use SPDK internally to maximize per-node throughput while implementing features like replication and erasure coding at the user-space level.
SPDK Components
- bdev — Block device abstraction layer. Backends include NVMe, malloc, aio, and Ceph.
- nvmf_tgt — NVMe-oF target application supporting TCP and RDMA transports.
- vhost — Exposes NVMe-oF storage to QEMU virtual machines via vhost-user.
- iSCSI target — Legacy protocol support using the same high-performance backend.