Chapter 27: Containerization and Local Orchestration

1. Containerisation and local orchestration

The news aggregator from Chapter 26 runs on a single machine and depends on PostgreSQL being installed locally. This chapter packages it as a Docker image, orchestrates the whole stack (API + Postgres + Redis) with Docker Compose, and adds Redis caching in front of expensive endpoints. We profile the API first so the caching win is measured, not assumed.

What you'll learn
  • How to profile a real API before adding infrastructure: timing middleware, identifying the slowest endpoints, load-testing locally
  • What containers actually are (and how they differ from virtual machines) so the rest of the chapter has a working mental model
  • Docker fundamentals: writing a Dockerfile, building an image, running and inspecting a container, multi-stage builds for smaller production images
  • Layer caching: ordering Dockerfile instructions so a code change doesn't invalidate the dependency-install layer
  • Docker Compose for multi-service orchestration: one YAML file defines the API, Postgres, and Redis as one runnable stack
  • Redis as a response cache: the decorator pattern, cache-key design, and how to measure the hit-rate against the baseline
What you'll build
  • middleware/timing.py — request-timing middleware for the pre-container profiling step
  • Dockerfile — multi-stage build that produces a small production image of the news aggregator
  • docker-compose.yml — API + Postgres + Redis in one orchestrated stack
  • .dockerignore — keep secrets and build artefacts out of images
  • redis_cache.py + main.py updates — Redis-backed response cache wired into expensive endpoints via a @cached decorator
  • requirements.txt update — add the Redis client without polluting the production dependency list

Why we reach for containers here

Chapter 26's news aggregator runs fine on the laptop where you built it. It breaks the first time you try to run it somewhere else: a different Python version on a teammate's machine, a missing system library on the CI runner, a Postgres install with a different default port on the staging server. Containers package the app with everything it needs to run -- Python, system libraries, the API code, the entrypoint -- into one image that runs the same on every machine that has Docker.

That property pays back twice in this chapter. First, the API is now a portable artefact: build once on your laptop, run anywhere. Second, the same Compose file that orchestrates Postgres + Redis + the API on your laptop is the artefact a teammate uses on day one (docker compose up) and the shape we will productionise in Chapter 28's cloud deployment.

Profile first, cache second

Reaching for Redis before profiling is a common mistake: caching slow endpoints is high-leverage, caching fast ones adds infrastructure for no win. Section 2 adds timing middleware to the API and runs a small load test, so by the time we wire Redis in section 6 we know which endpoints earn the cache and which don't. The result is a measured speedup against a known baseline, not "feels faster".

How the chapter is laid out

  • Performance profiling. Timing middleware, identifying the slowest endpoints, a small local load test that gives us the baseline numbers.
  • Understanding containers. What a container actually is, how it differs from a VM, and when you need one.
  • Docker fundamentals. First Dockerfile, build, run, inspect; then multi-stage builds and layer-cache discipline for fast rebuilds.
  • Orchestrating with Docker Compose. One YAML file describes the API + Postgres + Redis stack; one command brings it up.
  • Adding Redis caching. A @cached decorator backed by Redis; wire it into the endpoints profiling identified; re-measure to confirm the win.
  • Review. Patterns to take forward, a quiz on the load-bearing decisions, and a forward pointer to the cloud deployment in Chapter 28.

Next, in section 2, we add request-timing middleware to the Chapter 26 API and find the endpoints worth caching.