Editorial illustration; it does not depict a specific production system.
Published on 18 October 2026.
An API call fails. The application sees a timeout, waits a moment and sends the same request again. That sounds sensible until hundreds of clients do it together. The service becomes even busier, a payment or order is accidentally submitted twice, and a short incident turns into a larger outage.
Retries are useful, but they are not a universal fix. A good retry policy is a small reliability design: it recognises which failures might be temporary, gives the other service room to recover, and avoids repeating an operation that may already have succeeded.
Quick Answer
Before adding a retry loop, answer these questions:
- Is the error likely to be temporary, such as a timeout, connection reset, rate limit or selected server error?
- Is the operation safe to repeat, or does it create a side effect?
- Does the response include a
Retry-Afterinstruction or another service-specific limit? - What is the maximum number of attempts and total time the user can tolerate?
- What should the application do when all attempts fail?
Use a short timeout for each attempt, exponential backoff between attempts, random jitter to spread clients out, and a hard limit on attempts. For writes, use an idempotency key or another server-side mechanism that makes a repeated request produce one intended result. Log the attempt number and final outcome so you can see whether the policy helps or simply hides a dependency problem.
1. Start by classifying the failure
Not every non-success response deserves another request. A 401 or 403 usually means the credentials or permission must be fixed. A 400 means the request is invalid. Sending either one again will not improve the input or grant access.
Transient failures are different. A connection can reset, a load balancer can time out, or a service can return a temporary 500, 502, 503 or 504. A 429 Too Many Requests response can also be retryable, but the client should respect the server’s rate limit instructions when they are supplied.
This is not a rule to copy blindly. Read the API documentation and understand which errors are safe to repeat. A service may use a particular status code for a permanent business failure, or it may return a structured error code that is more useful than the HTTP status alone.
The important distinction is between “the request probably did not reach the service” and “the service rejected the request.” A network timeout does not prove that the server did nothing. It may have completed the work just before the connection disappeared.
2. Put a timeout on every attempt
Without a timeout, a retry policy can leave a request waiting for minutes and then start another request while the first one is still consuming resources. That is not resilience; it is uncontrolled accumulation.
Set a timeout that matches the operation. A search suggestion may need a fraction of a second. A report export may legitimately take longer, but it still needs a deadline and a way to continue asynchronously if the job is large.
Keep two limits in mind:
- The timeout for one attempt
- The total deadline for the whole operation
For example, an application might allow a user-facing profile lookup one second overall, while a background synchronisation job gets thirty seconds. The retry policy should not quietly exceed the caller’s deadline. When the deadline is reached, return a clear failure or queue the work for later instead of keeping the user waiting.
3. Add exponential backoff
Immediate retries create a tight loop precisely when a dependency is unhealthy. Exponential backoff increases the wait after each failed attempt. A simple sequence might be 200 milliseconds, 400 milliseconds, 800 milliseconds and 1.6 seconds, subject to a maximum.
The exact numbers are not universal. A rate-limited API may require a longer delay, while an interactive request may have no useful reason to try four times. The point is to avoid hammering a recovering service and to make the waiting behaviour predictable.
AWS describes retries, backoff and jitter as separate parts of a reliable client design. Its guidance also warns that retries can amplify load when many layers each retry the same call. Decide where the policy lives. If the SDK already retries, adding another loop in your controller may multiply attempts unexpectedly.
4. Add jitter so clients do not retry together
Imagine a thousand workers receive a timeout at the same moment and all wait exactly one second. They will send a second wave together. If that wave fails, they will create another synchronised wave two seconds later.
Jitter adds a random component to the delay. Instead of waiting exactly 800 milliseconds, each client might choose a delay within a bounded window. The service then sees a more gradual stream of retries rather than a spike.
For a small application, the algorithm does not need to be exotic. One reasonable pattern is to calculate an exponential ceiling and choose a random delay between zero and that ceiling. Cap the delay so a single request does not become invisible for too long, and record the chosen delay when diagnosing behaviour.
5. Treat writes differently from reads
Retrying a GET is often easier to reason about because the operation is expected to read data. It can still be expensive, and a read may trigger server-side work, but the intended result is not normally “create another thing.”
Writes need more care. A request to create an order, charge a card, send an email or reserve stock may succeed even when the client receives no response. Retrying it without protection can create a duplicate.
This is where idempotency matters. The client sends a unique idempotency key for one logical operation. The server stores the result associated with that key and returns the same result when it receives a safe repeat. The key must be scoped and retained long enough for the client to recover from the uncertainty. The API documentation should define how this works; do not invent a client-only convention that the server ignores.
If the API has no idempotency support, consider a different workflow: create a client-generated operation ID, check the operation status before creating a new one, or ask the user to confirm that an ambiguous action should be investigated. “Try again” is not a safe answer when money, inventory or notifications are involved.
6. Honour Retry-After and service limits
When a service sends Retry-After, treat it as meaningful input. It may specify a number of seconds or a time at which another request is acceptable. Combine that instruction with your own total deadline and maximum delay.
Do not assume that increasing retries will solve throttling. AWS’s API guidance makes the important point that retries can help a client recover from a temporary throttle, but they do not reduce the number of calls being made. If the application is consistently over the limit, lower the request rate, batch work, cache safe results or request an appropriate quota change.
Also protect your own service from callers. Rate limits, queues and concurrency limits prevent one slow dependency from consuming every worker and connection in your application.
7. Keep retries at one clear layer
A browser may retry a request, an SDK may retry it, a service client may retry it again, and a job queue may retry the whole job. Each layer can look reasonable in isolation while the combined system sends far more requests than expected.
Choose an owner for the policy. Put low-level transport retries in a shared client when they are safe and consistent. Put business-level retries in a workflow when the application needs to inspect state between attempts. Document the decision so another developer does not add a second loop because the first one was hard to find. A fallback provider still needs a limit and a test before the primary service fails.
8. Return a useful result when retries end
After the final attempt, do not return a generic “something went wrong” if the caller can act on a more specific result. A user-facing application might say that the change could not be confirmed and advise checking the account before trying again. A background job might place the item in a dead-letter queue with the operation ID.
Avoid pretending success because a retry loop ended without throwing an exception. For a write, distinguish between confirmed success, confirmed rejection and unknown outcome. That third state is essential when the connection failed after the server may have processed the request.
A small example
The following pseudocode shows the shape of a policy. It is intentionally incomplete: the real implementation must use the HTTP client’s cancellation and timeout features, parse service-specific errors and protect sensitive log data.
deadline = now + 5 seconds
attempt = 0
while now < deadline and attempt < 3:
attempt += 1
response = call_with_timeout(request, 1 second)
if response is successful:
return confirmed_success(response)
if response is a permanent client or permission error:
return confirmed_failure(response)
if request is a write without idempotency protection:
return unknown_outcome(response)
delay = service_retry_after(response) or random_delay(attempt)
wait_until(deadline, delay)
return retry_exhausted()
Common mistakes to avoid
Do not retry every exception, use a fixed delay for every client, or let attempts grow without a deadline. Do not hide the original error, log full access tokens or request bodies, or count a timeout as proof that no side effect occurred.
Do not add retries before measuring the dependency. If the service is returning permanent validation errors, the right fix is in the request. If it is consistently slow, investigate capacity, queries and timeouts rather than making every caller wait longer.
My Take
The best retry policy is deliberately boring. It has a small number of attempts, a visible deadline, a little randomness and a clear answer for ambiguous writes. Reliability improves when the client knows when to stop.
I also prefer testing retry behaviour with a fake dependency that returns timeouts, 429 responses and delayed successes. A happy-path unit test cannot show whether a thousand workers will retry in a synchronised burst or whether a payment can be submitted twice.
Bottom Line
Retries can turn a brief network problem into a smooth recovery, but only when they are selective and bounded. Classify the failure, time out each attempt, back off with jitter, respect service instructions and protect side-effecting operations with idempotency. When the client still cannot know what happened, report that uncertainty honestly and give the next layer a safe way to investigate.