mmap lets the OS page data in on demand via the page cache
Memory-mapped storage maps a file directly into the process address space. Instead of reading the file with explicit read calls and holding the data in a heap-allocated buffer, the process accesses the file through pointers, and the operating system handles paging. When the process touches a page that is not resident, the kernel takes a page fault, loads the page from disk into the page cache, and resumes the access. When memory pressure rises, the kernel evicts pages that have not been touched recently. From the application's perspective the file appears to be one large contiguous array, but only the pages that are actually being used occupy physical RAM. This is what allows Qdrant to serve a collection whose vectors and graph are much larger than the machine's RAM: the working set - the hot pages that queries actually touch - is what needs to fit in memory, not the whole dataset.
The mechanism has several consequences that matter for performance. First, there is no explicit load phase: startup is fast because the process does not need to read the entire dataset into memory, it just maps the files. Second, the page cache is shared across processes on the same machine, so multiple Qdrant processes or threads reading the same files do not double the memory usage. Third, latency becomes a function of cache hit rate rather than a fixed value: hot data returns in microseconds, cold data blocks on a disk read that can be tens of milliseconds on spinning disks or hundreds of microseconds on NVMe. This is the fundamental trade-off - you get the ability to serve a huge collection, but you lose control over what is in memory and you accept variable latency. Fourth, mmap interacts well with quantization: you can keep the small quantized vectors resident in RAM (always_ram=True) while the full-precision vectors stay on disk, so the graph traversal is fast and only the rescoring step touches disk. That hybrid is often the best configuration for large collections.
No explicit load: the OS pages data in on demand, so startup does not require reading the whole dataset.
Shared page cache: multiple readers of the same file share physical pages, so memory is not duplicated.
Variable latency: hot pages are fast, cold pages block on disk I/O; p99 is dominated by cache misses.
Hybrid patterns: quantized vectors in RAM for traversal, full-precision vectors on disk for rescoring, graph on disk for very large collections.
Eviction is not under application control: the OS decides what to evict, which can cause latency cliffs under memory pressure.
The trade-off is RAM cost against latency predictability. In-memory collections have low, stable latency but require enough RAM for the whole dataset plus the graph plus payload. mmap collections can run on a fraction of that RAM but their latency depends on cache hit rate and on what else is competing for memory on the machine. I choose mmap when the collection is much larger than RAM, when the query rate is low enough that a cold miss is tolerable, when cost matters more than tail latency, or when the collection is a cold tier that is queried infrequently. I keep things in RAM for latency-critical high-QPS paths. The common mistake is assuming mmap makes disk as fast as RAM. It does not - it makes disk tolerable when the working set fits in cache. If your working set does not fit, you will see latency cliffs as the cache thrashes. The second common mistake is ignoring the page cache when sizing a machine: if you size RAM for the vectors but forget the graph and payload, the cache will thrash under load even though the numbers looked fine on paper. The alternative to mmap is explicit disk I/O with application-level caching, which gives you control but is more code and usually performs worse because the OS cache is well-tuned. Version note: the on_disk flags on vector params, HNSW config, and quantization config have evolved across releases, and in some versions the mmap behavior is implied by these flags rather than exposed as a separate setting - check how your version exposes it.
Version-dependent: the on_disk flag has moved between vector params, HNSW config, and quantization config across releases, and the default values differ. In some versions, mmap is the implicit behavior for segments above memmap_threshold, controlled by the optimizer rather than by a per-vector flag. If you are designing around mmap, verify both the per-vector flags and the optimizer thresholds on your version, because the effective storage mode is the intersection of the two.
0-2 years experience
2-5 years experience
5-8 years experience
8+ years experience
