Design a three-stage retrieval pipeline using dense retrieval, sparse retrieval, fusion, and ColBERT reranking. What does each stage contribute?

The three stages are: (1) parallel first-stage retrieval with dense and sparse retrievers, (2) fusion of the two candidate lists into a single ranked list, and (3) late-interaction reranking of the fused candidates. Each stage has a distinct job. The dense retriever captures semantic similarity - it retrieves documents that mean the same thing as the query even when they share no words, which is what you want for paraphrases and conceptual matches. The sparse retriever (typically BM25 or a learned sparse model like SPLADE) captures lexical overlap - it retrieves documents that contain the query terms, which is what you want for exact matches, rare terms, proper nouns, and code identifiers. Neither alone is sufficient: dense retrieval misses exact term matches, sparse retrieval misses semantic paraphrases. Fusion combines the two ranked lists into one, typically with Reciprocal Rank Fusion (RRF), which is robust to score-scale differences between the two retrievers. The fused list is then reranked by a late-interaction model, which applies token-level matching to reorder the candidates. The reranker is the precision stage; the retrievers are the recall stages.

The mechanism of each stage determines what it can and cannot fix. Dense retrieval with HNSW gives sub-linear search over millions of documents, but its recall is bounded by the embedding model's ability to represent the query and by the ANN approximation. Sparse retrieval with an inverted index is exact at the term level, but its recall is bounded by vocabulary mismatch - if the user's query uses different words than the document, it will miss. Fusion mitigates both: RRF gives a document credit for appearing high in either list, so a document that is semantically relevant but lexically different (only in the dense list) and a document that is lexically identical but semantically different (only in the sparse list) both make it into the fused candidate set. The reranker then applies a much more expensive model to order the candidates correctly. Critically, the reranker cannot recover documents that neither retriever surfaced - the fused recall is the ceiling on end-to-end recall. This is why the candidate set sizes for the two retrievers and the fusion limit are the most important tuning parameters in the pipeline.

The trade-offs are latency budget allocation and candidate set sizing. Each stage adds latency, and the reranker is the most expensive per candidate, so the pipeline has to be tuned as a whole: the dense and sparse prefetch limits determine the fused candidate set size, which determines the reranker cost. A common configuration is 100 dense + 100 sparse, fused to 50, reranked to 10. But the right numbers depend on the corpus and the query distribution. The common mistake is to over-invest in one stage - usually the reranker - while under-investing in the retrievers, and then wonder why recall plateaus. If the fused recall is 0.85, no reranker can push end-to-end recall above 0.85. The second common mistake is using a naive score-normalization fusion (e.g. adding cosine similarity to BM25 score) without accounting for the fact that the two scores live on different scales; RRF avoids this by using ranks instead of scores. The third mistake is assuming that sparse retrieval is obsolete because dense retrieval is better on benchmarks. On real production queries with rare terms, product names, or error codes, sparse retrieval is often the difference between finding the answer and not. Version note: Qdrant's prefetch and fusion API supports RRF and DBSF, and the exact nesting semantics have evolved across releases - verify the shape of the query for your version.

Version-dependent: the nested prefetch with FusionQuery and the multivector rerank field is part of the qdrant-client 1.10+ API. On older versions, multi-stage retrieval had to be orchestrated in the application layer with multiple round trips. The in-engine version has lower latency and avoids serializing intermediate results, but the API shape is version-specific. Also note that sparse vector support and the exact fusion options (RRF vs DBSF) have evolved; check the release notes for your version before designing around a specific fusion mode.