Chapter 4: Safe and Reliable API Requests

1. When simple isn't enough

In this chapter you'll turn Chapter 3's four-line API call into code that doesn't fall over the moment something goes wrong: code that checks every response before trusting it, reads headers to know what came back, and handles rate limits, network errors, and binary data instead of crashing on them. By the end you'll have a Make → Check → Extract pattern you can apply to every API request you write from here on.

Chapter 3 showed you that talking to an API can be as short as four lines of Python. That code works. It also only works most of the time, and this chapter is about the difference.

Here's the snippet we'll keep coming back to:

Fragile API call 
import requests

response = requests.get("https://catfact.ninja/fact")
data = response.json()

print(data["fact"])

It's clean. It's simple. And on a fast connection at your desk, the call is reliable. But it carries four silent vulnerabilities that only show up in production:

  • No timeout. If catfact.ninja is slow, your program hangs forever waiting for a response that may never come.
  • No status check. If the server returns a 503 error page, the next line tries to parse HTML as JSON and crashes.
  • No content check. If the response isn't JSON at all, response.json() throws a JSONDecodeError.
  • No key check. If the API changes and renames the field, data["fact"] throws a KeyError.

The APIs you used in Chapter 3 were intentionally friendly: stable servers, predictable JSON, designed for learners. Real-world APIs are less polite. Servers return errors, networks drop requests, responses arrive in shapes you didn't expect, and a request that worked yesterday can behave differently today. The more your code assumes everything will go right, the more fragile it becomes.

The shift we're making in this chapter is less technical than psychological. Amateur code assumes success and crashes when that assumption breaks. Professional code assumes things will occasionally fail, plans for it, and keeps running. That's what users actually rely on.

Over the next seven sections you'll turn the fragile snippet above into code that handles real-world conditions gracefully: it'll catch network failures, recognise bad responses before parsing them, distinguish different kinds of errors, and still deliver something useful to the caller when something goes wrong.

What you'll learn
  • Validate responses before processing data, so failures surface early instead of silently
  • Read response headers to detect content types and handle different formats appropriately
  • Distinguish network errors, HTTP errors, and data-quality issues, and respond to each
  • Apply the Make → Check → Extract pattern consistently to every API request
  • Work confidently with binary responses like images, not just JSON text
  • Build applications that degrade gracefully instead of crashing when an API misbehaves
What you'll build
  • fetch_json.py — a defensive JSON fetcher applying Make → Check → Extract; refactored mid-chapter to use raise_for_status()
  • test_pattern.py — deliberate failure tests that exercise each error path of fetch_json_safely
  • inspect_headers.py — reader that pulls Content-Type, Content-Length, and rate-limit headers
  • check_rate_limits.py — rate-limit header parsing with backoff awareness
  • text_vs_binary.py — comparison of text and binary response accessors
  • save_image_wrong.py — deliberate text-mode mistake for binary data
  • save_image.py — corrected binary-write version of the image save
  • downloader.py — binary image downloader with content-type validation and safe write-mode handling
  • use_downloader.py — small caller that unpacks the downloader's return tuple

First we'll see what happens when you skip these checks entirely, then build the pattern that prevents it. Onwards.