Every payload index consumes memory and slows down writes, because each upsert must update the index for every indexed field. For a collection with many fields, indexing all of them can multiply the memory footprint and the write cost without providing any benefit for fields that are never filtered on. The right approach is to base the indexing decision on actual query patterns: index the fields that appear in production filters, and leave the rest unindexed. A field that is only returned in the payload but never filtered on does not need an index. A field that is filtered on rarely can be left unindexed if the latency cost of scanning is acceptable. A field that is filtered on frequently and selectively should be indexed with the type that matches the filter. The decision is empirical: instrument the application to log which fields are used in filters, aggregate over a representative period, and index the top fields by frequency and selectivity.

The mechanism that makes indexing expensive is that a payload index is a data structure separate from the payload itself. A keyword index maps each distinct value to a posting list; a datetime index stores sorted timestamps with min/max statistics; a full-text index stores an inverted index of terms. Each of these has its own memory footprint, and each must be updated on every write that affects the indexed field. For a collection with many fields and a high write rate, the index maintenance cost can dominate the write path. The benefit of an index is that it allows the planner to prune segments and to resolve the filter efficiently during traversal, which reduces query latency for filtered queries. The trade-off is therefore: index a field if the query latency benefit exceeds the memory and write-cost penalty. For frequently filtered selective fields, the benefit is large; for rarely filtered or non-selective fields, it is small or negative.

1
Index cost: memory footprint plus per-write update cost for each indexed field.
2
Index benefit: faster filtered queries via segment pruning and efficient filter resolution.
3
Index selectively: base the decision on actual query patterns, not on defensive indexing.
4
Measure usage: instrument the application to log which fields are filtered on and how often.
5
Consider selectivity: an index helps most for selective filters on high-cardinality fields.
6
Revisit periodically: query patterns change; re-evaluate which fields are indexed.
7
Drop unused indexes: if a field is no longer filtered on, drop its index to reclaim memory.

The trade-off is between the cost of indexing and the cost of not indexing. Without an index, a filtered query has to evaluate the filter for every candidate the traversal visits, which is slower but not catastrophic for infrequent queries. With an index, the query is faster but the index consumes memory and slows writes. The right balance depends on the query and write rates and on the latency SLO. The common mistake is to index every field defensively, which wastes memory and slows writes for no benefit. The second mistake is to index a field that is filtered on but with the wrong index type (e.g. keyword index for a date range), which does not provide the expected speedup. The third mistake is to never revisit the indexing decisions, so the collection accumulates indexes for fields that are no longer used. The fourth mistake is to assume that an index is always faster - for a low-cardinality field that matches most points, the index adds overhead without reducing the candidate set. Version note: the supported payload index types and the index options (e.g. is_tenant, on_disk) have evolved across Qdrant releases. Verify the index types available on your version and benchmark the impact of adding or dropping an index before making a decision.

javascript

Version-dependent: the payload index types and the API for creating and dropping them have evolved across Qdrant releases. The is_tenant flag, the on_disk option for payload indexes, and the support for different field schemas are recent additions. If you are tuning index coverage, benchmark the impact on your version with your data, because the cost and benefit of an index depend on the internal implementation.

Difficulty: 5/10

Topics: Payload Indexes, Schema Design, Best Practices

Scenario Questions

0-2 years experience

1
A teammate indexes every field defensively. Explain the cost and how you would decide which fields to keep.
2
You filter on a field that is not indexed and the query is slow. Explain the trade-off between adding an index and accepting the latency.

2-5 years experience

1
You have 20 payload fields and only 5 are filtered on. Describe how you would decide which to index and how you would monitor the decision over time.
2
Adding an index to a field speeds up queries but slows writes. Describe how you would measure both effects and make a decision.

5-8 years experience

1
Design a process for reviewing payload indexes quarterly, including the metrics you would collect and the criteria for adding or dropping an index.
2
You need to reduce the memory footprint of a collection by 20 percent. Describe how you would identify candidate indexes to drop and what you would measure before and after.

8+ years experience

1
You are designing a system where the query patterns evolve over time. Describe the architecture and the tooling that automatically identifies indexing opportunities and removes unused indexes.
2
Derive the break-even point where adding a payload index is worth the memory and write-cost overhead, as a function of query frequency, selectivity, and the latency SLO.

Follow-up Questions

❓How would you measure the memory and write-throughput cost of a payload index in production, and how would you decide whether it is worth keeping?
❓If a field is filtered on frequently but has low cardinality, is an index worth adding? Explain the reasoning.