How would you distinguish a latency problem caused by disk I/O from one caused by CPU-bound distance computation?

The two bottlenecks have different signatures and different fixes. Disk I/O latency is characterized by high disk utilization, elevated iowait, and variable latency that correlates with page-cache misses and with the size of the working set. CPU-bound latency is characterized by high CPU utilization on the search threads, stable latency that scales with ef and the number of candidates, and no correlation with disk activity. The first step is to look at the system metrics: on Linux, use iostat or vmstat to check disk utilization and iowait, and top or the equivalent to check CPU utilization of the Qdrant process. If disk utilization is near 100 percent and iowait is high, the bottleneck is I/O. If CPU is pegged and disk is idle, the bottleneck is compute. If both are moderate and latency is still high, the bottleneck may be elsewhere (network, lock contention, or the optimizer).

The mechanism behind each bottleneck is different and produces different workload-shape signatures. Disk I/O latency is determined by page-cache hit rate and disk speed, so it is sensitive to the working set size, the recency of queries, and the size of the collection relative to RAM. A query whose data was recently touched returns quickly; a query that touches cold pages pays the disk latency. This produces a bimodal latency distribution where the p50 may be fine and the p99 is bad. CPU-bound latency is determined by the number of distance computations, which scales with ef and the number of segments traversed, so it is sensitive to query parameters and to the number of segments, not to recency. This produces a tight latency distribution that shifts up or down with the parameters. To distinguish them, vary the workload: if increasing ef makes latency worse, it is CPU-bound; if increasing ef makes little difference but touching more of the collection makes latency worse, it is I/O-bound. You can also run a query against a small collection that fits in RAM and compare with the same query against the large collection - if the small collection is fast and the large one is slow, the difference is cache or disk.

The trade-off in fixing the bottleneck is cost and complexity against latency. Disk I/O problems are fixed by adding RAM to expand the page cache, by quantizing to shrink the working set, by moving to NVMe for lower per-read latency, or by exploiting locality with hot/cold tiering. CPU-bound problems are fixed by reducing ef (at the cost of recall), reducing the number of segments, adding cores, or offloading to a GPU for the distance computations. Each fix has its own cost profile, and the right one depends on which bottleneck you have. The common mistake is to assume the problem is CPU-bound because the Qdrant process is using CPU, when the CPU is spent on serialization, merging, or payload lookups rather than on distance computations. The second mistake is to add CPU cores when the bottleneck is I/O, which does nothing. The third mistake is to ignore the optimizer, which competes with queries for both CPU and I/O and can make a single bottleneck look like both. Version note: the metrics Qdrant exposes and the internal behavior of the search path have changed across releases, so use the metrics available on your version and correlate them with client-side latency rather than relying on a fixed diagnostic recipe.

Version-dependent: the metrics exposed by Qdrant and the internal search path have changed across releases. Some versions expose per-query timing or per-stage metrics; others do not. If you are diagnosing a bottleneck, use the system-level tools that are available on the host and correlate them with client-side latency, rather than relying on a Qdrant-specific metric that may not exist in your version.