What is the practical difference between storing vectors in-memory versus on-disk in a collection configuration, and when would you choose on-disk?

Storing vectors in-memory means they are allocated on the heap and always resident in RAM. Distance computations during HNSW traversal and rescoring hit memory directly, with no page faults and no disk I/O. This gives the lowest and most predictable latency, but it requires enough RAM to hold the full-precision vectors, the HNSW graph, and the payload indexes. Storing vectors on-disk means they are memory-mapped and paged in on demand. RAM usage drops dramatically - only the working set needs to be resident - but latency becomes a function of cache hit rate, and cold reads add tens of microseconds to tens of milliseconds per access. The practical difference is therefore RAM cost against latency predictability, and the right choice depends on which of those is the binding constraint for your workload.

The decision is not binary, because Qdrant lets you mix storage modes per component. You can keep the HNSW graph in RAM but the raw vectors on disk, so traversal is fast and only rescoring touches disk. You can quantize the vectors and keep the quantized versions in RAM while the full-precision versions stay on disk, so traversal uses the in-RAM quantized vectors and rescoring fetches the raw vectors for a small candidate set. You can put everything on disk for a cold tier. The most common production pattern for large collections is the hybrid: quantized vectors in RAM, graph in RAM if it fits or on disk if it does not, raw vectors on disk. This gets you close to in-memory latency for the traversal phase while keeping the RAM footprint proportional to the quantized data rather than the full float32 data. The full-precision vectors are only touched during rescoring of a small candidate set, so the disk I/O is bounded and predictable.

I choose on-disk when the collection is much larger than the machine's RAM, when the query rate is low enough that occasional cold reads are acceptable, when the workload is a cold tier that is queried infrequently, or when cost pressure makes RAM the dominant expense. I keep things in memory when the collection fits comfortably and the workload is latency-critical, or when the p99 SLO is tight enough that variable disk latency would violate it. The common mistake is assuming on-disk always means slow. With binary quantization, inline storage of quantized vectors in graph nodes, and rescoring over a small candidate set, an on-disk collection can serve queries in single-digit milliseconds - it is not the same as a naive disk-backed search. The second common mistake is sizing RAM by counting vectors only. The graph and payload indexes can dominate memory on large collections, so a collection whose raw vectors fit in RAM may still not fit once the graph is added. The third mistake is ignoring NVMe versus spinning disk: on-disk is a very different proposition on NVMe (hundreds of microseconds per read) than on HDD (tens of milliseconds). Version note: the on_disk flags and the interaction between vector, graph, and quantization storage have changed across releases, and the defaults differ - always inspect the effective config.

Version-dependent: the on_disk flag exists on vector params, HNSW config, and some quantization configs, but its exact behavior and defaults have changed across releases. In some versions, the optimizer's memmap_threshold determines whether a segment is stored on disk regardless of the per-vector flag, which means the effective storage mode is a combination of the two. If you need deterministic storage behavior, set both the per-vector flags and the optimizer thresholds explicitly, and verify with get_collection after creation.