At billion-point scale, how would your indexing and sharding strategy differ from a design that works fine at ten million points?

At ten million points, a design that keeps the full-precision vectors in RAM, uses a single HNSW graph per shard, and shards across a handful of nodes works fine. At a billion points, that design breaks in several ways. First, the RAM required to keep a billion 768-dim float32 vectors is about 3 TB, plus the graph (~100-160 GB) and the payload indexes. That is not feasible on a single node and expensive even across many nodes. The first structural change is quantization: binary or scalar quantization reduces the vector footprint by 4-32x, making the working set fit in a smaller, cheaper deployment. The second change is on-disk storage with inline quantized vectors: the full-precision vectors live on NVMe, the quantized vectors are in RAM, and the graph is on disk with the quantized vectors inline to halve the I/O per traversal. The third change is sharding: a billion points requires many more shards (e.g. 64-256) to distribute the data and the query load, and the shard count must be planned generously because it is fixed at creation. The fourth change is the query path: the traversal must be efficient enough that the per-query cost does not grow with the collection size, which means using quantized traversal with rescoring over a small candidate set. The fifth change is the indexing strategy: building a billion-point HNSW graph takes hours to days, and it must be done in a way that does not block ingest or queries.

The mechanism that makes these changes necessary is that the resources that are abundant at 10M points (RAM, a single machine, a single graph) become scarce at 1B points. The RAM ceiling is the most immediate: a single machine cannot hold a billion full-precision vectors, so quantization and on-disk storage are mandatory. The CPU ceiling is the second: traversing a billion-point graph with full-precision distances is too slow, so quantized traversal is mandatory. The shard ceiling is the third: a single shard with a billion points has a graph and payload indexes that are too large for a single node, so sharding across many nodes is mandatory. The build ceiling is the fourth: building a single graph over a billion points takes too long, so the build must be parallelized across shards and segments. The operational ceiling is the fifth: at a billion points, the optimizer, the backups, and the reindexing all take significant time, so they must be planned and scheduled. The design that works at 10M points does not have to worry about these ceilings; the design at 1B points is shaped by them.