5. Response headers in depth
Response headers are the instruction label on the package. They tell you what's inside, how to handle it, and where it came from, and ignoring them is a reliable source of bugs that are genuinely hard to track down.
Here's the kind of bug that eats an afternoon:
import requests
# Download user avatar
response = requests.get("https://api.example.com/user/avatar", timeout=5)
# Save as JPEG
with open("avatar.jpg", "wb") as f:
f.write(response.content)
# Result: a PNG wearing a .jpg name. Many viewers sniff the bytes
# and open it anyway, but anything that trusts the extension breaks:
# upload validators, batch processors, web servers setting Content-Type
# from the extension. The API returned a PNG, not a JPEG.
The fix? Check the Content-Type header and use the correct file extension:
import requests
response = requests.get("https://api.example.com/user/avatar", timeout=5)
# Check what we actually got
content_type = response.headers.get("Content-Type", "") # "image/png"
# Use correct extension
extension = "png" if "png" in content_type else "jpg"
with open(f"avatar.{extension}", "wb") as f:
f.write(response.content)
# Result: File opens correctly!
Three lines, an afternoon of debugging saved. That's what it looks like when a header does real work for you.
The headers worth knowing
The HTTP spec defines hundreds of headers, but you'll reach for the same small core set day in and day out. They all sit in response.headers, which behaves like a dictionary. Save this as inspect_headers.py to see the most important ones in action:
import requests
response = requests.get("https://httpbin.org/json", timeout=5)
print("Essential Headers:")
print("=" * 50)
# Content-Type: What format is the response?
content_type = response.headers.get("Content-Type", "unknown")
print(f"Content-Type: {content_type}")
print(" → Tells you: Is this JSON, HTML, an image, or something else?")
# Content-Length: How big is the response?
content_length = response.headers.get("Content-Length", "unknown")
print(f"\nContent-Length: {content_length} bytes")
print(" → Tells you: How much data to expect (useful for progress bars)")
# Server: What software handled your request?
server = response.headers.get("Server", "unknown")
print(f"\nServer: {server}")
print(" → Tells you: What's running on the other end (rarely useful)")
# Date: When was this response generated?
date = response.headers.get("Date", "unknown")
print(f"\nDate: {date}")
print(" → Tells you: Server's timestamp (useful for caching)")
# Connection: Should the connection stay open?
connection = response.headers.get("Connection", "unknown")
print(f"\nConnection: {connection}")
print(" → Tells you: Whether to reuse this connection (handled by requests)")
Run python inspect_headers.py and you'll see something like:
Essential Headers:
==================================================
Content-Type: application/json
→ Tells you: Is this JSON, HTML, an image, or something else?
Content-Length: 429 bytes
→ Tells you: How much data to expect (useful for progress bars)
Server: gunicorn/19.9.0
→ Tells you: What's running on the other end (rarely useful)
Date: Sun, 15 Sep 2025 18:30:00 GMT
→ Tells you: Server's timestamp (useful for caching)
Connection: keep-alive
→ Tells you: Whether to reuse this connection (handled by requests)
The one that matters most: Content-Type
Of all of them, Content-Type is the one you can't afford to ignore. It tells you which accessor to use on the response body, and mismatching the two is a reliable way to produce parse errors, corrupted files, or crashes:
application/jsongoes throughresponse.json().text/htmlgoes throughresponse.text.image/png,image/jpeg, and any other image type go throughresponse.content.application/pdfgoes throughresponse.content.
Rate-limit headers
Many APIs expose rate limits right on the response, so your code can slow itself down before getting blocked. The convention varies per API, but the common names all start with X-RateLimit-. Save this as check_rate_limits.py and point it at GitHub, which advertises its limits cleanly:
import requests
def check_rate_limits(url):
"""Check if an API provides rate limit information."""
response = requests.get(url, timeout=5)
print(f"Status: {response.status_code}")
print("\nRate Limit Headers:")
# Common rate limit headers (varies by API)
rate_headers = [
"X-RateLimit-Limit", # Total requests allowed
"X-RateLimit-Remaining", # Requests left in this window
"X-RateLimit-Reset", # When the limit resets
"Retry-After", # How long to wait (if rate limited)
]
found_any = False
for header in rate_headers:
value = response.headers.get(header)
if value:
print(f" {header}: {value}")
found_any = True
if not found_any:
print(" (No rate limit headers found - this API may not expose them)")
return response
# Test with GitHub API (they provide rate limit headers)
print("GitHub API Rate Limits:")
print("=" * 50)
check_rate_limits("https://api.github.com/users/octocat")
Running it produces something like this (your numbers will differ):
GitHub API Rate Limits:
==================================================
Status: 200
Rate Limit Headers:
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 57
X-RateLimit-Reset: 1778010141
Sixty requests per hour, fifty-seven remaining, and the limit resets at that Unix timestamp. The example reset value, 1778010141, converts to 2026-05-05T19:42:21+00:00; your value will be different. Well-behaved clients watch these numbers and slow themselves down before hitting zero.
The minimum rule to live by is simple: if you see X-RateLimit-Remaining: 0, stop sending requests until the reset time, and if you get a 429 with a Retry-After header, wait exactly that many seconds before trying again. APIs that enforce rate limits will block clients that don't respect them, so don't be one of those clients. Chapter 9 returns to this topic with a proper automatic-backoff strategy. For now, the binary question of "am I still allowed to call?" is enough to protect you. The next section takes us from text responses to binary data, where the rules change.