4. HTTP-Level Mocking

HTTP-level mocking is a fake post office for your tests. Your code thinks it's sending a real request to OpenWeather, Spotify, or GitHub; a mocking library intercepts the letter at the door, checks the address and contents match what the test expects, and hands back a canned reply. No real network, no rate limits, no flaky connections, no live credentials.

The rest of this section works through a short experiment. We'll meet Python's standard mocking tool, unittest.mock, apply it to a real HTTP client, watch it silently miss a bug, then reach for a library that catches that class of bug by design.

Python's built-in mocking tools

Python's standard library ships unittest.mock, a module for replacing real code with controlled fakes during tests. Two tools from it, patch and Mock, turn up in almost every pytest tutorial, so it's worth seeing what each one does before we use them.

patch temporarily replaces something (a function, method, or attribute) with a fake for the duration of one test. Here's a minimal example that patches requests.get so no real HTTP call leaves the machine:

from unittest.mock import patch
import requests

@patch("requests.get")
def test_api_call(mock_get):
    mock_get.return_value.status_code = 200

    response = requests.get("https://api.example.com")

    assert response.status_code == 200

The decorator swaps out requests.get inside the test. No real HTTP request leaves the machine. The object returned by the fake is itself a Mock, which is why mock_get.return_value.status_code = 200 works.

Mock is the other half of the story: a flexible fake object where attributes and method return values are configured by assignment. Here's one on its own, outside a test:

from unittest.mock import Mock

fake_response = Mock()
fake_response.status_code = 200
fake_response.json.return_value = {"data": "test"}

Set an attribute, get that attribute back. Set .return_value on a method, get that value when the method is called. That's the entire API. Used together, patch swaps out the thing you don't want to run and Mock hands back whatever the test needs.

That's everything most pytest tutorials teach about mocking. Next we'll apply it to a real HTTP client, and watch it quietly stop being enough.

The client we're testing

We'll test a small client for the OpenWeather API. It makes one GET request, parses the JSON, and returns three fields. Save this at the root of your project as weather_client.py (next to requirements.txt and .gitignore, not inside tests/):

import requests

def get_weather(city, api_key):
    response = requests.get(
        "https://api.openweathermap.org/data/2.5/weather",
        params={"q": city, "appid": api_key, "units": "metric"},
        timeout=5,
    )
    response.raise_for_status()
    data = response.json()
    return {
        "city": data["name"],
        "temp_c": data["main"]["temp"],
        "description": data["weather"][0]["description"],
    }

Now we'll test it. The standard first move is to patch requests.get with unittest.mock and assert that the function returns the right dict. Save this as tests/test_weather_client_with_mock.py:

from unittest.mock import patch, Mock
from weather_client import get_weather

def test_get_weather_transforms_response():
    fake_response = Mock()
    fake_response.status_code = 200
    fake_response.json.return_value = {
        "name": "Dublin",
        "main": {"temp": 11.2},
        "weather": [{"description": "light rain"}],
    }

    with patch("weather_client.requests.get", return_value=fake_response):
        result = get_weather("Dublin", api_key="secret_key")

    assert result == {
        "city": "Dublin",
        "temp_c": 11.2,
        "description": "light rain",
    }

Run it from the project root:

$ pytest tests/test_weather_client_with_mock.py
============================= test session starts =============================
collected 1 item

tests/test_weather_client_with_mock.py .                                [100%]

============================== 1 passed in 0.04s ==============================

Green. Now let's introduce a bug. In weather_client.py, rename the query parameter "appid" to "api_key". That's a plausible refactor mistake: OpenWeather will reject the request with a 401 in production. Rerun the same command:

$ pytest tests/test_weather_client_with_mock.py
tests/test_weather_client_with_mock.py .                                [100%]

============================== 1 passed in 0.04s ==============================

Still green. The test didn't notice. patch(requests.get) replaces the function regardless of what arguments it's called with, so the params dict and the URL are never inspected. The JSON transformation is still correct, so the assertion still holds, and the test passes while production is broken.

unittest.mock mocks Python objects, not HTTP traffic. The difference is where the interception happens: unittest.mock swaps out a Python function, while an HTTP-level mock intercepts at the transport layer. That matters because the URL, method, query string, and body of the real outgoing request are now all inspected. A typo in a parameter name, a wrong header, a missing endpoint slash: all caught immediately. That strictness is the feature. It turns the wire format itself into part of your test, which is what catches the bugs unittest.mock silently lets through.

Mocking with responses

The responses library intercepts HTTP calls at the transport layer. Register the URL, method, and response body you expect; any unmatched request fails with a ConnectionError. Install it:

pip install responses pytest requests

Now let's rewrite the same test with responses. This time we're checking two things: the JSON transformation, and the shape of the outgoing HTTP request. Save this as tests/test_weather_client.py (a fresh file alongside your existing test_weather_client_with_mock.py):

import responses
from weather_client import get_weather

@responses.activate
def test_get_weather_sends_correct_request():
    responses.add(
        responses.GET,
        "https://api.openweathermap.org/data/2.5/weather",
        json={
            "name": "Dublin",
            "main": {"temp": 11.2},
            "weather": [{"description": "light rain"}],
        },
        status=200,
    )

    result = get_weather("Dublin", api_key="secret_key")

    # Response transformation is correct
    assert result == {
        "city": "Dublin",
        "temp_c": 11.2,
        "description": "light rain",
    }

    # The outgoing HTTP request was shaped correctly
    assert len(responses.calls) == 1
    request_url = responses.calls[0].request.url
    assert "q=Dublin" in request_url
    assert "appid=secret_key" in request_url
    assert "units=metric" in request_url

Run the new test from the project root:

$ pytest tests/test_weather_client.py
tests/test_weather_client.py .                                          [100%]

============================== 1 passed in 0.05s ==============================

Now reintroduce the appid → api_key bug in weather_client.py and run the same command again:

$ pytest tests/test_weather_client.py
tests/test_weather_client.py F                                          [100%]

================================== FAILURES ===================================
_________________ test_get_weather_sends_correct_request __________________

ConnectionError: Connection refused by Responses - the call doesn't match
any registered mock
  - GET https://api.openweathermap.org/data/2.5/weather

============================== 1 failed in 0.06s ==============================

The bug unittest.mock missed is the first thing responses catches. The error message names the request that didn't match, which is usually enough to find the regression in seconds.

A quick note on the @responses.activate decorator you've been using: it installs the interception for the duration of one test. If you prefer an explicit context manager, with responses.RequestsMock() as rsps: does the same job.

Testing error paths

Real APIs fail in specific ways: 401 on expired tokens, 429 on rate limits, 503 on outages. Each failure gets its own short test. First fix the bug you introduced a moment ago (restore "appid" in weather_client.py), then add these two tests to the bottom of your existing tests/test_weather_client.py:

import pytest
import responses
import requests
from weather_client import get_weather

@responses.activate
def test_get_weather_raises_on_unauthorized():
    responses.add(
        responses.GET,
        "https://api.openweathermap.org/data/2.5/weather",
        json={"cod": 401, "message": "Invalid API key"},
        status=401,
    )

    with pytest.raises(requests.HTTPError) as exc_info:
        get_weather("Dublin", api_key="wrong_key")

    assert exc_info.value.response.status_code == 401


@responses.activate
def test_get_weather_raises_on_service_outage():
    responses.add(
        responses.GET,
        "https://api.openweathermap.org/data/2.5/weather",
        status=503,
    )

    with pytest.raises(requests.HTTPError):
        get_weather("Dublin", api_key="any_key")

Run the whole file from the project root. You should now see three tests passing: the happy path, plus the two error paths you just added.

$ pytest tests/test_weather_client.py
tests/test_weather_client.py ...                                        [100%]

============================== 3 passed in 0.05s ==============================

The pattern scales: register a status code and body, assert that your code reacts. No need to take down a real service to exercise the error path. In a production client you'd usually wrap get_weather in a retry-on-503 policy and test that as well.

When Mock is still the right tool

Use responses when your code calls requests.get/post/... directly. Use unittest.mock when your code calls an SDK that doesn't go through requests, such as most AWS clients, google-cloud libraries, or gRPC stubs. The rule: mock at the boundary that matters for the behaviour you're testing.

One exception worth knowing: moto is the AWS-specific equivalent of responses. It intercepts boto3 at the HTTP level, so you get the same strictness guarantees for S3, DynamoDB, SQS, and friends.