The WAL is an append-only log of operations on a shard. Every upsert and delete is serialized into the WAL and flushed to disk before, or in parallel with, being applied to the in-memory and on-disk segment state. Its primary job is durability: if the process crashes between applying a write to memory and flushing it into a segment, the write would be lost without a log. On restart, Qdrant replays the WAL from the last known flush point and reconstructs the state that was not yet persisted into segments. Without a WAL, an upsert acknowledged to the client could silently disappear on a crash, which is exactly the failure scenario it protects against.

The WAL has a second, equally important role in a distributed cluster: it is the stream that replicates writes from a shard's primary to its replicas. Each replica consumes the WAL and applies the same operations in the same order, which is what keeps replicas consistent without shipping full segments. This is the mechanism behind the consistency levels. When a write is acknowledged depends on write consistency: with write_consistency_factor=1, the primary acks after it has the write locally; with majority or all, the primary waits for that many replicas to confirm they have consumed the WAL entry. The WAL is truncated only after the corresponding state has been flushed into a segment and all replicas have caught up, which is why wal_retain and wal_segments_ahead exist. The unit of truncation is a WAL segment, so the log is a sequence of files that are appended to and then deleted once their contents are durable in segments.

1
Durability: replays unflushed operations after a crash, so acknowledged writes are not lost.
2
Replication: the primary streams WAL entries to replicas, which is how replicas stay in sync.
3
Truncation: WAL files are deleted only after their operations are persisted in segments and consumed by all replicas.
4
Ordering: write_ordering controls whether the WAL is flushed before or after the write is applied, trading latency for durability guarantees.
5
Capacity: wal_capacity_mb sets the size of a single WAL segment, which bounds how much is replayed on restart.

The trade-off is durability against write latency. Fsyncing every operation is the safest but slowest; batching amortizes the fsync cost but widens the window in which an acknowledged write could still be lost if the OS buffer is lost. Qdrant exposes this through write_ordering and the wait flag on the write. The common mistake is assuming that a successful upsert response means the data is durable everywhere. It does not - it means the write was accepted, and whether it is durable on one replica, a majority, or all depends on the write consistency factor and the wait flag. A second common mistake is assuming the WAL protects against disk failure. It does not - the WAL lives on the same disk as the data. Protection against disk failure is replication and snapshots, not the WAL. The third mistake is ignoring WAL size. If the WAL is allowed to grow unbounded (e.g. because a replica is far behind), restart times and disk usage can blow up. Version note: the WalConfig fields (wal_capacity_mb, wal_segments_ahead, wal_retain) and the write_ordering enum have changed across releases, and the defaults are not the same in all versions - set them explicitly if you depend on the behavior.

javascript

Version-dependent: the WalConfigDiff fields and their defaults have changed. In some releases wal_segments_ahead and wal_retain have different names or semantics, and the write_ordering enum values have been extended. Also, the interaction between the WAL and the optimizer's flush schedule is version-specific - in some versions, segment flush is triggered by WAL size thresholds as well as by the optimizer. If durability is a hard requirement for your workload, benchmark the actual crash-recovery time on your version by killing the process mid-ingest and measuring how long replay takes.

Difficulty: 7/10

Topics: Write-Ahead Log, Durability, Replication

Scenario Questions

0-2 years experience

1
You upsert a point with wait=False and the process crashes immediately after. Is the point guaranteed to be there on restart? Explain.
2
A teammate says the WAL protects against disk failure. Correct them and explain what actually does.

2-5 years experience

1
You restart a node and it takes 20 minutes to come back up. Diagnose whether the WAL is the cause and propose two changes to reduce restart time.
2
Your replication lag grows during a traffic spike and disk usage climbs. Explain the relationship between the WAL, replication lag, and disk growth, and how you would mitigate it.

5-8 years experience

1
Design a durability configuration for a payments-adjacent search system where losing an acknowledged write is unacceptable. What write consistency, ordering, and wait settings do you use, and what is the latency cost?
2
You need to migrate a collection to a new node without downtime. Describe how the WAL and the replication stream are involved in the migration and how you verify no writes were lost.

8+ years experience

1
Derive the relationship between write latency, fsync batching, and durability window, and explain how you would choose the batch size for a workload with a hard durability SLA.
2
You are asked to guarantee zero data loss across a region failover. Design the replication and durability scheme, including the role of the WAL, and identify the failure modes that would still lose data.

Follow-up Questions

❓How would you measure the actual durability window of an upsert in your deployment, and what would you change to close it?
❓If a replica is down for hours and the WAL grows unbounded, what happens to restart time and disk usage, and how would you prevent it?