Compared with dense-only retrieval, hybrid search normally costs more in three places: indexing, storage, and query execution. You maintain both a dense vector index and a sparse index, store both representations, and execute two candidate-generation paths before fusion. The sparse representation can be compact when it has few non-zero entries, but its index still requires memory and maintenance. At query time, generating both representations and searching both branches increases CPU, I/O, and latency, especially when candidate limits are high. The benefit is better robustness across semantic and lexical query classes. My production approach is to measure marginal relevance gain per millisecond and per GB rather than assuming hybrid is always worth the overhead. A common mistake is comparing only top-1 relevance and ignoring p95/p99 latency, ingestion throughput, and infrastructure cost.

javascript

1
Indexing cost increases because the system maintains both dense and sparse retrieval structures and any associated payload indexes.
2
Storage increases because each point carries an additional sparse representation and its index structures, although sparse storage depends strongly on the number of non-zero entries.
3
Query latency usually increases because both branches execute and their candidate lists must be fused; candidate depth is a major tuning parameter.
4
Trade-off: hybrid can materially improve recall for exact terms and rare vocabulary. A simpler dense-only design may still be preferable when queries are mostly semantic, latency is extremely constrained, or the lexical gain is negligible.

Difficulty: 8/10

Topics: Performance trade-offs, Indexing and storage, Latency optimization

Scenario Questions

0-2 years experience

1
Your team adds sparse retrieval and search gets slower. What additional work is the system now performing?
2
Why should you compare quality as well as latency before deciding whether hybrid search is useful?

2-5 years experience

1
Hybrid search improves recall but doubles query CPU. What parameters would you tune before removing sparse retrieval?
2
Disk usage increases after adding sparse vectors. What components of the new design would you inspect?

5-8 years experience

1
A latency budget allows only 20 ms additional p95 for hybrid retrieval. How would you profile and optimize the dense, sparse, fusion, and reranking stages?
2
Your sparse index improves exact-match recall by 8% but lowers ingestion throughput by 35%. How would you decide whether the trade-off is acceptable?

8+ years experience

1
Design a capacity model for hybrid search covering vector/index memory, disk, CPU, ingestion throughput, and p95/p99 query latency.
2
Your business wants hybrid retrieval globally, but only 15% of queries materially benefit from lexical matching. How would you architect selective hybrid routing to control infrastructure cost?

Follow-up Questions

❓Which production metrics would you compare before enabling hybrid search?
❓How would you reduce hybrid-search latency without removing the sparse branch?