One node in a three-node Qdrant cluster crashes. What happens to reads and writes for shards that had a replica on that node?

What happens depends on the replication factor of each shard. If a shard had a replica on the crashed node and at least one other replica elsewhere, the surviving replica takes over serving reads and writes. For a shard whose primary was on the crashed node, the cluster promotes one of the surviving replicas to primary, and operations continue. For a shard whose replica (not primary) was on the crashed node, the primary continues as before and the cluster simply has one fewer replica until the node recovers. In both cases, there is a brief window during which the cluster detects the failure and, if necessary, elects a new primary, but reads and writes resume without manual intervention. If a shard had only one copy and that copy was on the crashed node, there is no surviving copy, and operations on that shard fail until the node recovers or the shard is restored from a snapshot.

The mechanism behind failover is that the cluster metadata, maintained through Raft, includes the shard-to-node placement and the primary/replica roles. When a node becomes unreachable, the cluster marks it as down, and the placement logic triggers a promotion for any shard whose primary was on that node. The surviving replica is promoted, and it continues to accept writes and serve reads. If the cluster cannot achieve a quorum for metadata operations, no further metadata changes can be committed, but existing shards can continue to serve traffic according to their current placement. This is the distinction between the metadata plane and the data plane: the metadata plane needs quorum, the data plane does not, and a cluster with a downed node can continue serving traffic for any shard that has a surviving replica. The availability of the collection as a whole therefore depends on the replication factor and on which specific shard's data was on the crashed node. With replication_factor=1, a node crash takes down every shard whose primary was on that node; with replication_factor=2, a single node crash leaves at least one copy of every shard; with replication_factor=3, the cluster can tolerate two simultaneous node failures.

The trade-off is between replication cost and availability. Higher replication factor means more storage and more network traffic for WAL streaming, but it means the cluster tolerates more failures. The common mistake is to deploy with replication_factor=1 because it is cheaper and then be surprised that a single node failure takes down a large fraction of the collection. The second mistake is to assume that a cluster with one downed node always continues serving all traffic; it does, but only for shards that have a surviving replica, and the shard-to-node placement is not always what the operator expects. The third mistake is to confuse metadata quorum with data availability. A cluster can lose metadata quorum and still serve existing data; conversely, a cluster can have metadata quorum and still have unavailable shards if those shards are unreplicated. Version note: the failover behavior, the promotion logic, and the exact window during which a shard is unavailable after a node crash have changed across Qdrant releases. Some versions have more sophisticated placement and faster failover; others have simpler behavior. If you rely on failover for your SLA, test it by killing a node in a staging cluster and measuring the impact.

Version-dependent: the cluster info API, the shard placement reporting, and the failover behavior have changed across Qdrant releases. In some versions the info endpoint returns a detailed peer list with shard assignments; in others it is more limited. The time it takes to detect a node failure and promote a replica has also changed. If you are designing for an availability SLA, test the failover behavior on your version rather than relying on a general description.