What's the difference between scaling Qdrant vertically (bigger node) and horizontally (more shards/nodes), and when does horizontal scaling stop paying off?

Vertical scaling means giving a node more CPU, RAM, and faster storage. Horizontal scaling means adding shards and nodes so that the work is partitioned across machines. Vertical scaling is simpler: there is no fan-out, no distributed merge, no consistency concern beyond replication, and query latency is bounded by one machine's capability. Its limit is the largest machine you can buy and the fact that cost grows superlinearly at the top end - the biggest instances cost disproportionately more per unit of RAM and CPU. Horizontal scaling removes the single-machine ceiling and can also improve availability because the failure domain is smaller, but it introduces fan-out: every query without a shard key hits every shard, each shard returns a local top-k, and the coordinator merges. The coordinator's work and the network cost grow with shard count, and the slowest shard bounds the query latency.

The mechanism that determines when horizontal scaling stops paying off is the balance between parallelism gain and coordination overhead. Adding shards parallelizes the search, so ideally latency drops as 1/N. But the coordinator has to send the query to N shards, collect N responses, and merge them, and the tail latency of the query is the maximum of the N shard latencies. If the shards are uniform and fast, the maximum is close to the mean, and fan-out is cheap. If there is any skew - a hot shard, a slow node, or a shard with more data - the maximum is much worse than the mean, and adding more shards makes the tail worse rather than better. At some point the coordinator itself becomes the bottleneck: merging N lists of size k is O(N*k) work, and at high N the merge dominates the latency. There is also a fixed per-shard overhead (network round trip, request serialization) that does not shrink with shard size, so very small shards have poor efficiency. The practical ceiling is usually somewhere in the tens of shards per query before the overhead outweighs the parallelism, though it depends heavily on latency budget and on whether queries can be routed to a single shard.

The trade-off is simplicity and predictable latency against scale and availability. I scale vertically until the machine is the bottleneck or until cost per unit of capacity turns sharply upward, then scale horizontally. Horizontal scaling pays off most when queries can be routed to a single shard via a shard key, because then the fan-out cost disappears and the parallelism is free. It pays off least when every query must fan out to all shards, because then the coordinator and the slowest shard become the bottleneck. The common mistake is assuming horizontal scaling always improves latency. It improves throughput much more reliably than latency, because latency is bounded by the slowest shard and by the merge. The second mistake is increasing shard count to fix a memory problem - sharding partitions data across nodes, so it does help with capacity, but it does not help if the bottleneck is query cost or coordinator work. The third mistake is ignoring the coordinator: at high shard counts, the node running the coordinator becomes hot, and that is often the first thing to fail. Version note: the shard placement and rebalancing behavior, the coordinator's implementation, and the default sharding strategy have changed across Qdrant releases, so the crossover point where horizontal scaling stops paying off is version-specific and should be measured rather than assumed.

Version-dependent: the sharding strategy, rebalancing, and coordinator behavior have changed across Qdrant releases, and the point at which horizontal scaling stops paying off depends on those internals. If you are planning a cluster, benchmark fan-out latency at the shard counts you are considering on your version, and measure the coordinator's CPU under load rather than extrapolating from a small cluster.