Why retries can become dangerous
A retry feels helpful in isolation. Under stress, many retries arrive at once and compete with the original traffic.
Interactive Systems Explainers
Retry Storm Playground
See retries amplify overload.
Key Observations
What to Notice
Timeout amplification
Shorter timeouts fail faster. Each failure can create more work before the old work has drained.
Positive feedback loops
More load creates more waiting. More waiting creates more retries. The loop can accelerate suddenly.
Why systems collapse
Once the service is buried, recovery lags behind. Retry traffic keeps circling even after new traffic calms down.
Real-World Context
Retries are built into HTTP clients, SDKs, queues, job runners, and service meshes because transient failures are normal. Trouble starts when many callers retry the same struggling dependency at the same time.
A retry policy can quietly become a traffic generator during the exact moment capacity is scarce. Backoff, jitter, retry budgets, idempotency, and circuit breakers are design choices that balance resilience against amplification. Poor defaults can spread the problem across every service using the same client library.
Interactive Systems Explainers