What is Inline Storage, and how does embedding quantized vectors directly into HNSW graph nodes improve disk-based search performance?

Inline storage is a Qdrant optimization for on-disk collections where the quantized vector data is stored directly inside the HNSW graph node, rather than in a separate location that has to be fetched with a second read. In a normal on-disk HNSW layout, each graph node contains its edges (neighbor IDs), and the vector data lives elsewhere in the file. When the search traverses the graph, it reads a node's edges, then for each candidate it reads the corresponding vector to compute the distance. That is two random reads per node visit, and on disk the second read is the expensive one because it is a separate I/O operation to a different location. Inline storage eliminates the second read: the quantized vector is right there in the node, so a single read gets both the edges and the vector. This turns the per-node cost from two random I/Os into one, which for an I/O-bound on-disk search is roughly a 2x improvement - often more, because the second read is the one that misses the page cache most often.

The mechanism depends on quantization being in play. You cannot inline a full-precision 768-dim float32 vector into a graph node without bloating the node enormously - that would multiply the size of every node and destroy the cache and I/O benefits. But a quantized vector is small: int8 is 768 bytes, binary is 96 bytes. Inlining the quantized vector adds a bounded amount to each node, and the distance computation during traversal uses the quantized vector anyway, so the full-precision vectors can stay in a separate store that is only touched during rescoring. This is why inline storage and quantization are designed together: quantization shrinks the vector enough to fit in the node, and inline storage removes the second I/O that quantization would otherwise require. The result is that on-disk HNSW with inline quantized vectors can approach the latency of in-memory search for large collections that do not fit in RAM, at the cost of the quantization accuracy loss (which is mitigated by oversampling and rescoring).