What is Inline Storage, and how does embedding quantized vectors directly into HNSW graph nodes improve disk-based search performance?

Inline storage is a Qdrant optimization for on-disk collections where the quantized vector data is stored directly inside the HNSW graph node, rather than in a separate location that has to be fetched with a second read. In a normal on-disk HNSW layout, each graph node contains its edges (neighbor IDs), and the vector data lives elsewhere in the file. When the search traverses the graph, it reads a node's edges, then for each candidate it reads the corresponding vector to compute the distance. That is two random reads per node visit, and on disk the second read is the expensive one because it is a separate I/O operation to a different location. Inline storage eliminates the second read: the quantized vector is right there in the node, so a single read gets both the edges and the vector. This turns the per-node cost from two random I/Os into one, which for an I/O-bound on-disk search is roughly a 2x improvement - often more, because the second read is the one that misses the page cache most often.

The mechanism depends on quantization being in play. You cannot inline a full-precision 768-dim float32 vector into a graph node without bloating the node enormously - that would multiply the size of every node and destroy the cache and I/O benefits. But a quantized vector is small: int8 is 768 bytes, binary is 96 bytes. Inlining the quantized vector adds a bounded amount to each node, and the distance computation during traversal uses the quantized vector anyway, so the full-precision vectors can stay in a separate store that is only touched during rescoring. This is why inline storage and quantization are designed together: quantization shrinks the vector enough to fit in the node, and inline storage removes the second I/O that quantization would otherwise require. The result is that on-disk HNSW with inline quantized vectors can approach the latency of in-memory search for large collections that do not fit in RAM, at the cost of the quantization accuracy loss (which is mitigated by oversampling and rescoring).

The trade-off is that inline storage increases the size of each graph node, which increases the memory (or disk) footprint of the graph itself. For an int8-quantized 768-dim vector, that is 768 bytes per node on top of the edges, which for a large collection is a non-trivial addition. The benefit is that the I/O reduction usually outweighs the size increase, but it is a real trade-off and depends on the ratio of vector size to edge size. For binary quantization the added size is tiny (96 bytes for 768 dims), so inline storage is almost always a win. For int8 it is still usually a win on disk, but the calculus is closer. The common mistake is assuming inline storage helps in-memory collections. It does not meaningfully - if the graph and vectors are in RAM, the second read was already cheap. Inline storage is a disk optimization, and its value is proportional to how I/O-bound the workload is. The second mistake is enabling it without quantizing, which either fails or bloats the graph depending on the version. Version note: inline storage and the associated configuration options were added in recent Qdrant releases, so verify availability on your version before designing around it.

Version-dependent: inline storage and the on-disk HNSW configuration options have evolved across recent Qdrant releases. The exact field names (on_disk on the vector params vs the HNSW config, and how inline storage is toggled) may differ on your version. Always check client.get_collection() after creation to confirm the effective configuration, and re-benchmark after upgrading, because the performance characteristics of on-disk search depend heavily on the storage layout that the current version produces.