More shards means more parallelism, but it also means more fan-out cost. Every query that is not routed to a single shard has to be sent to every shard, each shard runs its local search, and the coordinator collects the results and merges them into a global top-k. The coordinator's work grows linearly with the number of shards: it has to send N requests, receive N responses, and merge N lists. The network cost grows with the number of shards and the size of each response. And the latency is bounded by the slowest shard, not the average, so any skew in shard performance or load is amplified as the shard count grows. At small shard counts, the parallelism dominates and queries get faster. At large shard counts, the coordination overhead dominates and queries get slower, or the latency plateaus. The crossover point depends on the latency of a single shard, the coordinator's processing speed, and the network latency between the coordinator and the shards.

The mechanism behind this is the classic parallel-overhead trade-off. If a query takes T seconds on a single shard and is perfectly parallelized across N shards, the search time drops to T/N, but the coordination adds a fixed cost C plus a per-shard cost F. The total latency is roughly C + NF + T/N. This function has a minimum: as N increases, T/N decreases but NF increases, so there is an optimal N. Beyond that optimal N, adding more shards increases latency. The exact shape depends on the ratio of T to F. For a small collection where T is small, the fixed overhead dominates and sharding does not help. For a large collection where T is large, sharding helps until the coordination cost catches up. There is also a second effect: the coordinator itself becomes a bottleneck at high shard counts, because it has to process N responses and merge N lists, and its CPU or network bandwidth is finite. This is why very high shard counts (hundreds) rarely help a single-query latency, even though they help throughput.

1
Fan-out cost: N requests, N responses, and a merge that is O(N*k).
2
Slowest shard: latency is bounded by the maximum, not the mean, so skew is amplified.
3
Coordinator bottleneck: at high N, the coordinator's CPU and network become the limit.
4
Optimal N: the shard count that minimizes C + N*F + T/N, which depends on the workload.
5
Throughput vs latency: more shards can increase throughput even when they increase single-query latency.
6
Single-shard routing: queries with a shard key hit one shard and avoid fan-out entirely.
7
Custom sharding: the way to get the benefits of sharding without the fan-out cost for scoped queries.
8
Monitoring: per-shard latency, coordinator latency, and merge time to detect when N is too high.

The trade-off is between parallelism and coordination. The common mistakes are: (1) assuming that more shards always means faster queries, when at high N the coordination dominates; (2) not measuring the per-shard latency distribution, so a single hot shard is invisible until the p99 is bad; (3) choosing a shard count that is much higher than needed for the data, which adds overhead without benefit; (4) not using custom sharding for tenant-scoped queries, which would avoid the fan-out entirely; (5) forgetting that the coordinator is a single point for a given request and can become the bottleneck. Version note: the coordinator's implementation and the merge semantics have evolved across Qdrant releases. The prefetch/fusion API changed how multi-stage queries are planned and merged. The behavior at high shard counts may differ. If you are tuning shard count, benchmark the query latency at several shard counts on your version.

javascript

Version-dependent: the coordinator's merge semantics, the prefetch/fusion API, and the shard placement logic have changed across Qdrant releases. The crossover point where more shards stops helping is version-specific and depends on the workload. Benchmark on your version rather than relying on a general rule.

Difficulty: 8/10

Topics: Sharding, Query Execution, Scaling

Scenario Questions

0-2 years experience

1
You increase the shard count from 4 to 16 and queries get slower. Explain why and what you would do.
2
A teammate says more shards always means faster. Explain the fan-out cost with a concrete example.

2-5 years experience

1
Your p99 is much worse than your p50 on a 16-shard collection. Diagnose whether the cause is fan-out, a hot shard, or the coordinator.
2
You need to increase throughput without increasing latency. Describe the sharding strategy that achieves this.

5-8 years experience

1
Design a benchmark that measures the query latency at several shard counts and identifies the optimum for a given workload.
2
You need to support both scoped and global queries. Describe the sharding design that makes scoped queries cheap without making global queries unacceptable.

8+ years experience

1
Derive the optimal shard count as a function of the per-shard latency, the coordinator cost, and the QPS target. Where does the model break down?
2
You are designing a system that must serve global queries over a very large corpus with a low p99. Describe the architecture that avoids the coordinator bottleneck.

Follow-up Questions

❓How would you measure the optimal shard count for a specific workload, and what would you do if the optimum is not a power of two?
❓If the coordinator is the bottleneck at high shard counts, what architectural changes would you consider?