Python requestsTimeouts, Retries, and Backoff Done Right

The fix is not a bigger try/except. It is three habits that production code almost always has and tutorials almost always skip. A timeout on every call. Automatic retries for the failures worth retrying. And backoff, so those retries do not turn into an accidental denial-of-service attack on the very server you are trying to reach.

This guide walks through all three, ending with a reusable session you can lift into a project. It assumes Python 3.10 or later. The retry examples use urllib3 2.x because backoff_jitter was added in version 2.0. In a virtual environment for this guide, install Requests and a compatible urllib3 version:

python -m pip install "requests>=2.32,<3" "urllib3>=2,<3"

The examples use https://api.example.com/data as a placeholder. Replace it with a JSON endpoint you can access before running them.

What is the default timeout in Python requests?

Here is the call almost every tutorial starts with. It fetches some JSON and prints it.

import requests

response = requests.get("https://api.example.com/data")
print(response.json())

The default is timeout=None: Requests sets no time limit. If the server accepts the connection but never replies or closes it, your code can wait indefinitely. The operating system may still report a network failure, but Requests supplies no deadline of its own. Inside a web app, that can leave a worker blocked.

How to set connect and read timeouts

The fix is one keyword argument. Pass timeout to every request you make.

import requests

response = requests.get("https://api.example.com/data", timeout=10)
print(response.json())

timeout=10 applies ten seconds separately to the connect timeout and the read timeout. It does not give the whole call a ten-second deadline. To choose different values, pass a tuple:

# (connect timeout, read timeout)
response = requests.get(
    "https://api.example.com/data",
    timeout=(3.05, 27),
)

The first value limits each connection attempt to an IP address. The second limits how long a socket read can wait for data, including the first response bytes. (3.05, 27) is an example from the Requests timeout documentation, not a universal production setting. Choose values that fit the API and how long your caller can wait.

The one rule that always holds is this. Set a timeout on every single request, not most of them. A short connect timeout with a more generous read timeout is a sensible starting point, and you can raise the read value for endpoints you know are slow.

A response that keeps delivering data can take longer than the read timeout overall. Trying multiple IP addresses, retrying a request and waiting between retries can add more time. Neither a scalar nor a tuple guarantees a total elapsed-time limit; an application that needs a strict deadline must enforce it separately.

Why one attempt is not enough

A timeout turns an excessive wait for a connection or more data into a failure. Some failures are transient: a connection reset, a momentary DNS hiccup, or a load balancer returning 503 Service Unavailable while it shuffles traffic. Run the same request a moment later and it may succeed.

This is the case for retries. Not retrying everything blindly, but recognising that a whole category of network failures clears up on its own if you simply ask again after a short pause.

• Connection errors. The socket dropped or the server refused the connection. Often gone on the next try.
• Read timeouts. One slow response does not mean the next will be slow.
• Server errors in the 5xx range. A 502, 503, or 504 usually signals a temporary problem on their side, not yours.
• Rate limiting. A 429 Too Many Requests is the server explicitly telling you to wait and try again.

The distinction that matters is timing versus request. Retry the failures that are about timing. Do not retry the failures that are about your request, because a 404 Not Found or a 401 Unauthorized will return the same answer every time, and retrying it just wastes time and hammers the server.

The fragile hand-rolled retry

The obvious first instinct is to write the loop yourself. It looks reasonable.

import time
import requests

for attempt in range(5):
    try:
        response = requests.get("https://api.example.com/data", timeout=10)
        response.raise_for_status()
        break
    except requests.exceptions.RequestException:
        time.sleep(2)
else:
    raise RuntimeError("All attempts failed")

It works, but it is doing a lot of things slightly wrong. It retries every error, including the 404s and 401s that will never recover. It waits a flat two seconds each time instead of backing off. It has no jitter, so ten machines running this code retry in lockstep. And this is the simple version. Add per-status rules, a growing delay, and a cap, and you have quietly reimplemented a library that already ships inside requests, only with more bugs. The retry logic in urllib3 is battle-tested and plugs in with a few lines, which turns maintaining retry code into simply configuring it.

The right way: HTTPAdapter and Retry

The proper tool is built from three pieces you mount together once. A Retry object describes the policy, an HTTPAdapter applies that policy to outgoing requests, and a Session carries the adapter so every call through it inherits the behaviour.

import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

retry = Retry(
    total=5,
    backoff_factor=0.5,
    backoff_jitter=0.1,
    status_forcelist=[429, 500, 502, 503, 504],
    allowed_methods=["GET", "HEAD", "OPTIONS"],
    raise_on_status=False,
)

adapter = HTTPAdapter(max_retries=retry)

session = requests.Session()
session.mount("https://", adapter)
session.mount("http://", adapter)

response = session.get("https://api.example.com/data", timeout=(3.05, 27))
response.raise_for_status()
print(response.json())

Calls through session now use this retry policy for eligible failures. We mount the adapter for both URL schemes. Notice that timeout still lives on the individual request: it limits connection and read waits within attempts, while the retry policy controls additional attempts and their spacing.

The adapter can retry a read timeout while waiting for response headers, but it does not replay a download that stalls after those headers arrive. Requests reads the body outside the adapter's retry loop, so your code must still handle failures while consuming the response.

Create the session once and reuse it for many requests. Beyond carrying the retry policy, a Session reuses the underlying TCP connection across calls to the same host, which is faster than opening a fresh connection every time. Creating a new session per request throws that benefit away.

Backoff: spacing out the retries

The single most important setting above is backoff_factor. Without it, retries fire back to back, and if the server is already struggling, a wall of immediate retries is the last thing it needs. Backoff makes each retry wait a little longer than the one before.

With backoff_factor=0.5, the first retry fires immediately, and the growing pause kicks in from the second retry onward. After that, the delay roughly doubles each time. This is exponential backoff, and it gives a struggling server room to recover instead of piling on.

The backoff_max setting, 120 seconds by default, caps a computed backoff delay. It does not cap a server's Retry-After instruction, so it is not a ceiling on every possible wait.

There is one more failure mode worth designing for. If dozens of machines run the same code and all fail at the same instant, pure exponential backoff makes them all retry at the same instant too. That is the thundering herd. The example adds up to 0.1 seconds of randomness with backoff_jitter to each nonzero computed delay. The immediate first retry remains immediate. See the urllib3 Retry reference for the policy settings.

What to retry, and what not to

Two settings decide which failures get a second chance. Getting them right is what separates a smart retry policy from one that wastes time on hopeless requests.

status_forcelist picks the status codes worth retrying

Listing [429, 500, 502, 503, 504] explicitly enables retries for these statuses on allowed methods. Without a status list, Retry can still retry 413, 429 and 503 when they carry Retry-After. A 404 or 401 stays out of our list; fix the URL or authentication rather than repeating the same request.

allowed_methods controls which HTTP verbs may retry

This is about idempotency. A GET can be repeated safely, because asking for the same data twice changes nothing. A POST often cannot, because it might create a second order or charge a card again. By default urllib3 allows status and read retries for idempotent methods, leaving POST out. Connection errors before a request is sent can still be retried regardless of this method filter.

When a server returns 429 or 503 with a positive Retry-After delay, urllib3 uses that delay in preference to exponential backoff, subject to the header limits in your installed version. This lets the server tell clients when to try again.

A reusable session factory

Rather than repeat the setup everywhere, we wrap it in one function. Save this as http_client.py at your project root and import it wherever you need a configured session.

import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry


def make_session(
    retries=5,
    backoff_factor=0.5,
    backoff_jitter=0.1,
    status_forcelist=(429, 500, 502, 503, 504),
):
    """Return a requests Session with sensible retry and backoff defaults."""
    session = requests.Session()

    retry = Retry(
        total=retries,
        backoff_factor=backoff_factor,
        backoff_jitter=backoff_jitter,
        status_forcelist=status_forcelist,
        allowed_methods=frozenset(["GET", "HEAD", "OPTIONS"]),
        raise_on_status=False,
    )

    adapter = HTTPAdapter(max_retries=retry)
    session.mount("https://", adapter)
    session.mount("http://", adapter)
    return session

Using it stays readable, and the only thing each call site has to remember is the timeout.

from http_client import make_session

session = make_session()

response = session.get("https://api.example.com/data", timeout=(3.05, 27))
response.raise_for_status()
data = response.json()

The function takes its retry count, backoff factor, jitter, and status list as arguments, so a noisy third-party API can get a more patient session while an internal service gets a stricter one. The shared defaults keep most call sites simple.

How do I set a timeout for a requests Session?

A standard Session has no built-in default-timeout setting. Assigning session.timeout does not change its requests. Pass a timeout on each call, or centralise repeated calls in a helper. For JSON GET requests, add this below make_session() in http_client.py:

def get_json(session, url, *, timeout=(3.05, 27)):
    response = session.get(url, timeout=timeout)
    response.raise_for_status()
    return response.json()

Import both helpers in the calling file. This call uses the helper's timeout; pass timeout=(3.05, 60) to override it for a slower endpoint:

from http_client import get_json, make_session

with make_session() as session:
    data = get_json(session, "https://api.example.com/data")

Only calls through get_json() inherit that default. A direct session.get() still needs its own timeout. You can check the distinction in the Session request parameters.

Knowing when to stop

Retries are not infinite, and they should not be. The total setting is the number of retries allowed after the first attempt, not the total number of HTTP attempts. With total=5, a request can be tried once and then retried up to five more times before the policy gives up. What happens then depends on raise_on_status. With it set to False, as above, the final response comes back to your code even if it is a 503, and you decide what to do with a bad status using raise_for_status(). With it left at its default of True, an exhausted retry on a forced status raises an exception instead.

Either way, the important habit is to treat a failed-after-retries call as a real outcome, not a surprise. The request layer has done its job. It tried, it backed off, it waited on Retry-After, and the failure that survived all of that is one worth handling deliberately.

That handoff is its own topic. Once a call has genuinely failed, the question becomes how to categorise the failure, read what the server actually said, and fail gracefully instead of crashing. Our companion guide, Handling API Errors in Python, picks up exactly there. And when you want to prove your retry and timeout logic behaves under failure without hitting a live server, the testing guide shows how to mock those conditions in a test suite.

When a dedicated library helps

The adapter approach is the right default for HTTP, because the retry happens at the transport layer and applies to every call through the session. But sometimes you want to retry something that is not a single HTTP request. A small block of code, a database write, a multi-step operation. For that, a decorator-based library is cleaner.

• tenacity wraps any function in configurable retry logic with its own backoff and stop conditions. Useful when the thing you are retrying is wider than one request.
• backoff offers a similar decorator with exponential backoff built in.
• httpx is an alternative HTTP client with async support. Its built-in transport retries cover connection errors and connection timeouts; for status-code retries such as 429 or 503, pair it with a general-purpose retry tool.

For ordinary API calls, the built-in HTTPAdapter and Retry are enough, and they keep your dependency list short. Reach for a dedicated retry library only when you need to wrap logic that is broader than a single HTTP call.

Frequently asked questions

Does Python requests retry failed requests automatically?

No. By default, requests makes a single attempt and does not retry anything, and it has no timeout either. Automatic retries come from mounting an HTTPAdapter configured with a urllib3 Retry object onto a Session. Once that is in place, every call through the session retries according to your policy without any loop in your own code.

What is a sensible default timeout for requests?

Requests defaults to timeout=None, which sets no timeout. An explicit tuple such as timeout=(3.05, 27) gives separate connect and read limits. Treat those numbers as a starting example and adjust them for the API and caller. They do not set a deadline for the whole call.

Does a requests Session have a default timeout?

No. A standard Session does not use an assigned session.timeout attribute. Pass timeout on each request or use a helper that supplies it, such as the JSON request helper above.

Should I retry POST requests?

Usually not. A GET is safe to repeat because it changes nothing, but a POST may create or modify data, so a blind retry can double-submit an order or charge a card twice. By default urllib3 leaves POST out of the retryable methods. Only retry a write if the operation is genuinely safe to repeat or the endpoint supports an idempotency key.