Suppose you request the top 10 nearest vectors from the entire collection and only afterward apply a filter such as tenant=A. If none of those ten nearest points belongs to tenant A, the application gets zero results even though thousands of tenant-A points may exist.

The root problem is that the top-k operation and the filter were applied in the wrong order. ANN returns a candidate set optimized for global similarity, not for the constrained result set. A highly selective filter can therefore remove most or all of the retrieved candidates.

The trade-off in a naive workaround is over-fetching. You can retrieve a much larger global candidate set and filter it afterward, but that increases latency and still does not provide a principled guarantee that enough eligible candidates were considered.

A common mistake is assuming 'top-k then filter' is equivalent to 'top-k among filtered points.' It is not. For production workloads, the filter should participate in the database retrieval process so the search can find eligible candidates rather than discarding them afterward.

javascript

1
Top-k global retrieval followed by filtering can under-return results
2
Highly selective filters make post-filtering especially dangerous
3
Over-fetching is a workaround, not a reliable solution
4
Filtering should participate in retrieval when correctness and scale matter

Difficulty: 6/10

Topics: Filtered vector search, ANN search, Filtering

Scenario Questions

0-2 years experience

1
There are 100,000 vectors in a collection but only 20 belong to tenant A. Why can searching the global top 10 and then filtering produce zero results?
2
A developer says the database should always find tenant-A results because they exist. What is wrong with that reasoning?

2-5 years experience

1
A filter returns zero results only for small top-k values but works when top-k is 1,000. What does that suggest about the query architecture?
2
Your team proposes retrieving 10,000 global neighbors and filtering them in application code. What trade-offs would you explain?

5-8 years experience

1
A production query has highly selective authorization filters and occasionally returns fewer results than requested. How would you determine whether the issue is post-filtering, ANN behavior, or insufficient eligible data?
2
A workaround increases candidate retrieval by 100x but causes latency spikes. What architectural change would you recommend instead?

8+ years experience

1
Your security model requires retrieval to be constrained by tenant and authorization filters while maintaining high ANN recall. How would you design the retrieval architecture to make post-filtering unnecessary?
2
A benchmark shows integrated filtering has higher latency than unfiltered ANN, but post-filtering violates result-count correctness. How would you establish the production trade-off?

Follow-up Questions

❓Why does increasing the initial top-k only partially solve post-filtering?
❓How does integrated filtering change the candidate-selection problem?