Evaluate this claim: 'Cosine similarity and normalized dot product always produce identical rankings.' What subtlety do candidates often miss here?

The claim is true only under a precise condition: both the query vector and every stored vector must be normalized to unit length. Cosine similarity is defined as dot(a, b) / (||a|| * ||b||). If both ||a|| and ||b|| are exactly 1, the denominator is 1 and cosine reduces to dot(a, b). The rankings are then identical. The subtlety that candidates miss is that this equivalence is conditional, and in practice the condition is often violated in ways that are invisible because the search still returns plausible results. The first violation is unnormalized query vectors: if the client normalizes documents at ingest but not queries, the query's norm is not 1, and cosine and dot will rank differently because the dot product scales with the query norm. The second violation is unnormalized stored vectors: if a single document's vector was not normalized (a bug in an ingest path, a partial batch, a manual upsert), that document's score is off by its norm and its ranking shifts. The third, more subtle violation is the storage metric: if the collection is created with Distance.DOT but the vectors were normalized, and the query is normalized, the rankings match cosine. But if the collection is created with Distance.COSINE, Qdrant normalizes at query time (or assumes normalized inputs), and the behavior differs from a DOT collection with unnormalized vectors. The fourth subtlety is that even when the rankings are identical in exact arithmetic, floating-point rounding can produce tiny differences that change the order of near-tied results, which matters for recall measurement but not for user experience.

The mechanism that makes this matter is that the equivalence is about the ordering of scores, not about the scores themselves. Cosine and dot produce the same ordering when both sides are normalized, but they produce different scores - cosine is bounded to [-1, 1] while dot is unbounded. If you compare a score threshold (e.g. 'only return results with similarity > 0.8'), the threshold means different things under the two metrics and can change which results are returned. This is why a system that switches from DOT to COSINE, or vice versa, can appear to 'work' while actually returning different top-k in edge cases. The practical implication is that the metric must be chosen deliberately and applied consistently: normalize everything and use DOT, or use COSINE and let the engine handle normalization, but do not mix. The other practical implication is that a test asserting on exact scores will fail when the metric changes even if the ranking is the same, so tests should assert on rankings, not on raw scores.