Disaggregated Storage
Disaggregated storage is an architecture where storage resources are physically separated from compute resources and shared across multiple hosts over a network. NVMe-oF is the primary protocol enabling disaggregated storage at flash-native latency, making it a cornerstone of modern cloud-native and Kubernetes infrastructure.
Disaggregated vs Converged (DAS) Architecture
In a converged (DAS) architecture, each server has its own local NVMe SSDs. Storage capacity is tied to server count — adding storage means adding servers even if you only need more capacity. In a disaggregated architecture, dedicated storage nodes hold NVMe SSDs and serve multiple compute nodes over the network.
Benefits of Disaggregated NVMe Storage
- Independent scaling — Add storage capacity without adding compute nodes (and vice versa). This reduces over-provisioning waste.
- High availability — Multi-path NVMe-oF connections and replication between storage nodes eliminate single points of failure that exist in DAS.
- Storage pooling — Multiple compute nodes share a common NVMe pool, improving utilization versus per-server silos.
- Kubernetes-native — CSI drivers expose disaggregated NVMe storage as PersistentVolumes. Pods get persistent block storage that survives node restarts without binding data to specific hosts.
Latency Trade-off
Disaggregation adds network latency. With NVMe-oF over TCP, this overhead is typically 20–25µs — bringing total latency to 25–40µs vs 5–20µs for local PCIe. For most workloads (databases, Kubernetes storage, AI training), this is imperceptible and the architectural benefits far outweigh the latency difference.