7. Chapter review

The receive-side patterns from this chapter compose into a single shape: accept the POST, verify the signature, persist the event with a delivery-ID dedup key, return 200 OK, let a background worker drain the queue and do the slow downstream work. The provider changes (GitHub, Stripe, Twilio), the destination changes (Slack, email, billing system, dashboard), and Slack appears in both columns because it sends webhooks too. The architecture carries. You also have a working GitHub Activity Notifier that runs end to end against ngrok in development and, per the Section 6 deployment paragraph, against Railway in production.

Key skills

  • Pull vs push. Articulate when polling is the right call (you control timing; cheap to fetch; not real-time-critical) and when push is (real-time user-facing notifications; rate-limited APIs; per-execution billing).
  • Webhook anatomy. Read a real webhook request top to bottom: URL routing, custom headers (X-GitHub-Event, X-GitHub-Delivery, X-Hub-Signature-256), JSON payload structure.
  • Production-shape receiver. Build a Flask endpoint that persists every delivery to SQLite with a unique delivery_id column, returns 200 within milliseconds, and tolerates retries idempotently.
  • HMAC signature verification. Use a shared secret + HMAC SHA-256 to verify sender authenticity (the request came from someone holding the secret). This is not body confidentiality -- the payload is still plaintext over HTTPS -- but it is the standard line of defence against forged requests.
  • Replay protection. Combine signature verification with the delivery-ID unique constraint to drop replayed-but-valid requests at the storage layer.
  • Background worker pattern. Decouple fast HTTP acknowledgement from slow downstream work using a separate process that drains a queue. The first step toward Celery/RQ/SQS in production.
  • Event normalisation. Turn provider-specific JSON into a small internal shape (kind, actor, repository, title, url, extra) so the rest of your code is provider-agnostic.

Chapter review quiz

Seven questions across the architectural calls the chapter makes. They pressure-test the trade-offs (when to poll, when to push, when HMAC actually helps, what the dedup column actually buys you) rather than recall.

Select question to reveal the answer:
You are designing a system that needs to know when a long-running OCR job finishes. The OCR provider supports both polling (poll /status/{job_id}) and webhooks (provider POSTs to your URL on completion). When is each the right call?

Polling wins when the job is short (seconds), the system is internal, and adding a webhook endpoint creates more operational surface than it saves. Webhooks win when jobs are long (minutes-to-hours), when polling burns through rate limits or per-execution billing, when latency-to-notification is user-facing, or when you would otherwise need many concurrent connections holding open. The Receipt Scanner in Chapter 21 ran synchronously -- requests.post() blocked until OCR.space returned -- because the jobs are short. Polling and webhooks are both ways to handle the long-running variant; Chapter 21 sketched polling at the end of receipt-scanner.njk, and this chapter builds the webhooks variant.

Your webhook handler does the Slack POST inline before returning 200 OK. What goes wrong, and which two architectural changes fix it?

The provider waits while your handler waits on Slack. If Slack is slow or down, your handler exceeds the provider's timeout (typically ~10 seconds for GitHub), the provider retries -- which means you Slack-post a second time when Slack eventually answers -- and after enough retries the provider may disable the webhook entirely. The two fixes are persist + acknowledge fast (store the event in SQLite, return 200 in milliseconds, never call Slack from the handler) and process in a background worker (a separate process drains the queue, retries on its own schedule, and is allowed to be slow).

GitHub retries a delivery you already processed. What does an idempotent handler do, and why is the delivery_id column the right dedup key?

The handler tries to insert the event, hits the UNIQUE constraint on delivery_id, catches the integrity error, and returns 200 immediately without doing any downstream work. delivery_id is the right key because the provider guarantees uniqueness per delivery -- two webhooks with the same body but different deliveries (legitimate duplicate events) will get different IDs and be processed normally, while two retries of the same delivery (the case dedup is for) share the ID. Hashing the body would conflate those two cases.

Why is the "HMAC SHA-256 verifies the message wasn't tampered with end to end" framing misleading, and what does signature verification actually prove?

HMAC verifies sender authenticity: the request came from someone holding the shared secret, and the body has not been altered between signing and your verification. It does not verify body confidentiality -- the payload is plaintext over HTTPS, and any TLS-terminating intermediary (Cloudflare, your load balancer, a logging proxy) sees it in the clear. It also does not provide non-repudiation in the cryptographic sense, because both sides hold the secret. The right mental model is "this request came from someone who knew the secret" plus "the body matches what they signed," nothing more.

You leak GITHUB_WEBHOOK_SECRET in a public commit. What is the correct response, and what specifically does an attacker who finds the secret gain?

Treat the secret as compromised: rotate it immediately. In GitHub repository settings, generate a new secret, update GITHUB_WEBHOOK_SECRET in your environment (Railway Variables tab, local .env, or wherever you store it), and redeploy. The attacker can forge requests that pass HMAC verification, so they can trigger your worker as if the events were real -- which means they can fan out Slack notifications, write rows into your database, or trigger any downstream side effect your worker performs. They cannot retrieve past payloads or impersonate GitHub on the wire (TLS protects the transport); the damage is bounded to "things your handler does on receipt."

Why does the chapter introduce a normaliser between the raw GitHub payload and the Slack-posting code, instead of letting the worker read GitHub fields directly?

Two reasons. (1) Provider isolation. If GitHub changes its payload shape -- not impossible across major API versions -- only the normaliser changes; everything downstream (Slack formatting, status updates, database writes) sees the same internal shape. (2) Multi-provider readiness. Adding a Stripe or GitLab feed later means writing a new normaliser that produces the same internal shape; the Slack code does not change. The cost is one small dictionary; the payoff is that the rest of your code stops caring about which provider an event came from.

Your ngrok URL changes every time you restart ngrok (free tier), so you re-register the webhook every dev session. In production this stops being a problem -- why, and what changes between dev and production?

Production uses a stable URL. When you deploy the receiver to Railway (per the deployment paragraph at the end of Section 6), Railway gives you a permanent your-app.up.railway.app domain. You register that URL in GitHub once and it stays valid as long as the service runs. ngrok is a development convenience -- public tunnel to your laptop -- not a production hosting strategy. The free-tier URL churn is fine because dev sessions are short; in production the receiver is a long-lived service with a fixed address, exactly the same shape as the Music Time Machine deployment in Chapter 20.

Looking forward

Chapter 22 covered the receive side: external services push events to your endpoint, your code stores them and reacts. Chapter 23, Asynchronous APIs and Performance Optimization, flips back to the outbound side but changes the timing model. Instead of one request at a time, you fan out many requests concurrently -- the difference between waiting 30 seconds for 50 sequential API calls and waiting 3 seconds for 50 parallel ones.

Python's asyncio and httpx.AsyncClient are the tools Chapter 23 uses, and the patterns compose naturally with what you built in this chapter. The background worker drains a queue serially today; the same worker with async I/O can drain it in parallel without spawning threads. The result is the production architecture that real backend services use: webhooks for inbound, async for outbound aggregation, both running in the same process.