How would you retrieve 500 candidates with dense vectors and rerank them using a ColBERT multivector?

I would make the dense vector search the high-recall candidate generator and set its limit to 500. The second stage would use the ColBERT multivector representation to score those candidates more precisely, returning only the final top-k results.

The reason for this architecture is computational economics. Dense ANN retrieval is comparatively cheap and can search a large collection, while late-interaction scoring is more expensive. Restricting ColBERT scoring to 500 candidates gives the reranker a manageable workload while preserving most of the candidate recall.

The important detail is that the ColBERT representation must already be stored as a compatible multivector in Qdrant, and the query must target that vector configuration. The exact query syntax for multivectors and late interaction is version-sensitive, so I would use the API corresponding to the deployed Qdrant release.

A common mistake is setting the prefetch limit equal to the final top-k. That leaves the reranker too few candidates and can cap recall before reranking even starts.