How would you architect a system to gracefully degrade - rather than fail outright - when a burst of traffic exceeds provisioned Qdrant capacity?

Question

Accepted Answer

Graceful degradation has four layers: admission control, request shaping, fallback execution, and circuit breaking. Admission control rejects or delays requests when the system is above a threshold, preventing the overload from cascading. Request shaping queues requests and processes them at a controlled rate, so the burst is absorbed rather than dropped. Fallback execution runs a cheaper version of the query when the full query is too expensive: a lower ef, fewer candidates, no reranking, a smaller result set. Circuit breaking stops sending requests to a dependency that is failing, giving it time to recover rather than hammering it. The combination lets the system serve a degraded but useful response under overload, rather than failing all requests or collapsing entirely. The design must define the degradation levels and the thresholds at which each kicks in, and it must be tested under load to verify that the degradation is graceful. The mechanism that makes each layer effective is that it reduces the load on the bottleneck resource. Admission control reduces the arrival rate. Queuing smooths the arrival rate. Fallback execution reduces the per-request cost. Circuit breaking reduces the load on a failing dependency. The key insight is that the degradation must be progressive: the system should not jump from full service to no service. It should have intermediate states - slightly reduced quality, then more reduced, then minimal - so that users get a useful response at every level. The thresholds must be based on measurable signals: queue depth, latency, error rate, CPU utilization. The fallback queries must be pre-defined and tested, because a fallback that has not been tested is not a fallback. The circuit breaker must have a recovery policy: when the dependency recovers, the system should return to full service gradually, not all at once, to avoid a second overload. Admission control: reject or delay requests when above a threshold. Queuing: absorb the burst and process at a controlled rate. Fallback query: lower ef, fewer candidates, no reranking, smaller result set. Circuit breaker: stop sending to a failing dependency; recover gradually. Progressive degradation: multiple levels, not a binary switch. Thresholds: based on queue depth, latency, error rate, CPU. Pre-defined fallbacks: tested and ready, not improvised. Recovery: return to full service gradually to avoid a second overload. The trade-off is between quality and availability. A system that degrades gracefully serves a worse but useful response under load; a system that does not degrades catastrophically, failing all requests. The common mistakes are: (1) no admission control, so the overload cascades; (2) no fallback queries, so the only option is to fail; (3) a fallback that has not been tested, so it fails when it is needed; (4) a circuit breaker that does not recover gradually, causing a second overload; (5) thresholds that are not based on measurable signals, so the degradation is unpredictable. Version note: the degradation strategies are application-level, but the Qdrant features that enable them - lower ef, fewer candidates, no reranking - are configured via the query parameters. The exact parameters and their effect have changed across Qdrant releases. from qdrant_client import QdrantClient, models
import time

client = QdrantClient('http://localhost:6333')

# Degradation levels: full, reduced, minimal.
def search(query_vector, filters, level='full'):
    if level == 'full':
        ef, limit, rerank = 128, 10, True
    elif level == 'reduced':
        ef, limit, rerank = 64, 10, False
    elif level == 'minimal':
        ef, limit, rerank = 32, 5, False
    else:
        raise ValueError(level)

hits = client.query_points(
        collection_name='docs',
        query=query_vector,
        limit=limit,
        query_filter=filters,
        search_params=models.SearchParams(hnsw_ef=ef),
    ).points
    if rerank:
        hits = rerank_cross_encoder(hits, query_vector)
    return hits

# Admission control and circuit breaker at the service layer.
class CircuitBreaker:
    def __init__(self, threshold=5, cooldown=30):
        self.failures = 0
        self.threshold = threshold
        self.cooldown = cooldown
        self.open_until = 0

def allow(self):
        if time.time() < self.open_until:
            return False
        return True

def on_failure(self):
        self.failures += 1
        if self.failures >= self.threshold:
            self.open_until = time.time() + self.cooldown
            self.failures = 0

def on_success(self):
        self.failures = 0

breaker = CircuitBreaker()

def serve(query_vector, filters):
    if not breaker.allow():
        return fallback_static_response()   # cached or static fallback
    try:
        result = search(query_vector, filters, level=current_level())
        breaker.on_success()
        return result
    except Exception:
        breaker.on_failure()
        raise V

How would you architect a system to gracefully degrade - rather than fail outright - when a burst of traffic exceeds provisioned Qdrant capacity?

Queuing, load shedding, cheaper fallback queries, and circuit breakers

How would you architect a system to gracefully degrade - rather than fail outright - when a burst of traffic exceeds provisioned Qdrant capacity?

Queuing, load shedding, cheaper fallback queries, and circuit breakers

Scenario Questions

Follow-up Questions