How would you decide, for a specific collection, whether to enable quantization with rescoring versus simply moving vectors on-disk without quantization?

Both options are ways to reduce the RAM footprint of a collection, but they trade against different things. Quantization with rescoring keeps a compressed representation of the vectors in RAM (or wherever you put it), so the traversal uses the small quantized vectors and only the final candidate set is rescored against the full-precision vectors. The trade is a small, measurable accuracy loss for a large reduction in memory, and the latency impact is usually modest because the rescoring step only touches a bounded number of candidates. Moving vectors on-disk without quantization keeps full accuracy but makes every vector access a potential disk read. The traversal itself then touches disk, which is much slower than touching RAM, and the latency becomes dominated by page-cache hit rate. The trade is latency and tail variance for exact accuracy and no quantization complexity.

The decision framework has three inputs. First, how sensitive is the application to accuracy? If a two-point recall drop would be invisible to users - most search and recommendation systems - quantization is the better choice. If the application is compliance search, exact-match retrieval, or anything where a missing result has a real cost, on-disk without quantization is safer. Second, how tight is the latency SLO? Quantization with rescoring tends to give a much better p99 because the traversal touches RAM; on-disk without quantization has a p99 that depends on disk latency and cache behavior, which is hard to bound. If the SLO is tight, quantization wins. Third, what is the query volume? Rescoring adds CPU work per query, so at very high QPS the rescoring cost may become the bottleneck, in which case on-disk without quantization or quantization without rescoring may be preferable. In practice the hybrid - quantized vectors in RAM for traversal and full-precision vectors on disk for rescoring - is often the best answer, because it combines the low latency of in-memory traversal with the low cost of on-disk storage, and it exposes the accuracy trade-off as a tunable knob (oversampling and rescoring).