Exponential backoff with jitter prevents the thundering herd problem by randomizing retry timings, ensuring that thousands of clients don't all attempt to reconnect simultaneously when a service recovers, thereby avoiding overwhelming the system.
The thundering herd problem occurs when a large number of clients or services are waiting for a resource or service to become available, and all attempt to reconnect or retry at exactly the same moment. This synchronized surge of requests can overwhelm the recovering service, causing it to fail again and creating a cycle of failures and recovery attempts. Exponential backoff with jitter solves this by introducing increasing delays between retries and adding randomness to those delays, spreading out requests over time and preventing synchronized stampedes.
Definition: Increases the wait time exponentially between retry attempts (e.g., 100ms, 200ms, 400ms, 800ms, 1600ms) .
Purpose: Gives the failing service time to recover before the next retry, reducing the retry rate over time .
Risk: Without jitter, all clients calculate the same wait times, leading to synchronized retry waves .
Definition: Adds random variation to the calculated backoff delay (e.g., wait_time * (0.5 + random.random()) or wait_time + random(0, wait_time)) .
Purpose: Spreads retry attempts across a time window instead of having all clients retry simultaneously .
Effect: The "herd" becomes a distributed, manageable stream of requests .
Full Jitter (AWS recommended): random_between(0, base * 2^attempt). Spreads requests across the full range from 0 to max backoff .
Equal Jitter: backoff/2 + random_between(0, backoff/2). Centers the distribution around half the backoff time .
Decorrelated Jitter: base * random_between(1, 3). Adds randomness proportional to the current backoff .
Consider a database outage that affects 10,000 microservice instances. Without backoff, all 10,000 instances would try to reconnect immediately, overwhelming the database and likely causing it to crash again . With exponential backoff, each instance would wait 100ms, 200ms, 400ms, etc. However, all would still attempt at the same intervals—creating synchronized spikes. Adding jitter randomizes the exact timing, so at the 100ms mark, only 10% of instances might reconnect, spreading the load over the next several seconds .
AWS SDK Default: The AWS SDK for JavaScript includes built-in exponential backoff with jitter for retryable errors .
Configuration: maxRetries: 3 and retryDelayOptions: { base: 100, customBackoff: ... } .
DynamoDB Best Practices: AWS recommends using exponential backoff with jitter for DynamoDB operations to handle throttling and temporary failures .
The combination of exponential backoff and jitter is essential for self-healing systems. Without these mechanisms, a temporary service outage can cascade into a sustained failure as retry storms prevent recovery. With them, the system gradually resumes normal operation with minimal disruption. This pattern is a cornerstone of resilient distributed systems, recommended by major cloud providers and used extensively in production environments.
0-2 years experience
2-5 years experience
5-8 years experience
8+ years experience
