9. Testing and verification

The four layers are now wired together; this section adds the test pyramid that catches regressions before they ship. Many fast unit tests at the base check individual functions in isolation; fewer integration tests in the middle verify the contracts between layers; a handful of end-to-end tests at the top exercise the complete workflow against the live API.

The point of the pyramid is the cost gradient. Unit tests run in milliseconds and run on every save. Integration tests run in seconds and run before every commit. End-to-end tests run in minutes and run before deploy. The shape isn't an aesthetic preference; it's the only way to get fast feedback without losing coverage.

Testing strategy

The three tiers map onto what each kind of test catches:

Test level What it tests Example Speed
Unit tests Individual functions and classes in isolation Input validator, error categoriser, data models Fast (ms)
Integration tests Layer interactions and data flow API client → processor, processor → validator Medium (seconds)
End-to-end tests Complete workflows from input to output Full weather dashboard request with real APIs Slow (seconds to minutes)

The next three sections build the three test files in order; each ends with the verification you'd run against it. Then a coverage pass reads the whole suite at once.

tests/test_components.py: pure functions in isolation

Unit tests exercise the validators, the error categoriser, and the dataclass constructors: anything that can be tested without making a network call. Save the file at tests/test_components.py and run it with pytest tests/test_components.py:

tests/test_components.py 
"""
Unit tests for weather dashboard components.

Tests individual functions and classes in isolation.
Fast, focused tests that run in milliseconds.
"""
import pytest
from datetime import datetime

# Import components to test
from input_validator import InputValidator
from errors import categorize_error
from validators import validate_structure, validate_range
from models import Location, CurrentWeather, WeatherData

class TestInputValidator:
    """Unit tests for input validation."""

    def setup_method(self):
        """Initialize validator before each test."""
        self.validator = InputValidator()

    def test_valid_city_name(self):
        """Test validation of simple city name."""
        result = self.validator.validate_location_input("London")

        assert result.normalized_query == "London"
        assert result.confidence >= 0.6

    def test_city_with_state(self):
        """Test validation of city with state code."""
        result = self.validator.validate_location_input("San Francisco, CA")

        assert result.normalized_query == "San Francisco, CA"
        assert result.confidence >= 0.8

    def test_coordinates(self):
        """Test validation of coordinate input."""
        result = self.validator.validate_location_input("51.5074,-0.1278")

        assert result.normalized_query == "51.5074,-0.1278"
        assert result.confidence >= 0.95

    def test_empty_input(self):
        """Test that empty input raises validation error."""
        with pytest.raises(Exception) as exc_info:
            self.validator.validate_location_input("")

        assert "cannot be empty" in str(exc_info.value).lower()

    def test_input_too_short(self):
        """Test that single character input raises error."""
        with pytest.raises(Exception) as exc_info:
            self.validator.validate_location_input("a")

        assert "too short" in str(exc_info.value).lower()

    def test_input_too_long(self):
        """Test that excessively long input raises error."""
        long_input = "a" * 101

        with pytest.raises(Exception) as exc_info:
            self.validator.validate_location_input(long_input)

        assert "too long" in str(exc_info.value).lower()

    def test_whitespace_normalization(self):
        """Test that whitespace is properly trimmed."""
        result = self.validator.validate_location_input(" London ")

        assert result.normalized_query == "London"

class TestErrorCategorization:
    """Unit tests for error categorization from Chapter 9."""

    def test_empty_input_categorized_as_user_input(self):
        """Test that empty input categorizes as user_input."""
        category = categorize_error(ValueError(), user_input="")

        assert category == 'user_input'

    def test_too_long_input_categorized_as_user_input(self):
        """Test that too-long input categorizes as user_input."""
        long_input = "a" * 101
        category = categorize_error(ValueError(), user_input=long_input)

        assert category == 'user_input'

    def test_network_timeout_categorized_as_transient(self):
        """Test that timeouts categorize as transient."""
        import requests
        error = requests.exceptions.Timeout()
        category = categorize_error(error)

        assert category == 'transient'

    def test_connection_error_categorized_as_transient(self):
        """Test that connection errors categorize as transient."""
        import requests
        error = requests.exceptions.ConnectionError()
        category = categorize_error(error)

        assert category == 'transient'

    def test_keyerror_categorized_as_not_found(self):
        """Test that KeyError (no results) categorizes as not_found."""
        error = KeyError('results')
        category = categorize_error(error)

        assert category == 'not_found'

class TestDataValidation:
    """Unit tests for validation utilities from Chapter 12."""

    def test_validate_structure_success(self):
        """Test structure validation with valid data."""
        data = {
            'main': {'temp': 15.3},
            'weather': [{'description': 'clear'}]
        }

        is_valid, error = validate_structure(data, ['main', 'weather'])

        assert is_valid is True
        assert error is None

    def test_validate_structure_missing_section(self):
        """Test structure validation detects missing sections."""
        data = {'main': {'temp': 15.3}}

        is_valid, error = validate_structure(data, ['main', 'weather'])

        assert is_valid is False
        assert 'weather' in error

    def test_validate_range_within_bounds(self):
        """Test range validation with valid value."""
        is_valid, error = validate_range(15.3, 'temperature', -100, 60)

        assert is_valid is True
        assert error is None

    def test_validate_range_below_minimum(self):
        """Test range validation detects values below minimum."""
        is_valid, error = validate_range(-150, 'temperature', -100, 60)

        assert is_valid is False
        assert 'below minimum' in error

    def test_validate_range_above_maximum(self):
        """Test range validation detects values above maximum."""
        is_valid, error = validate_range(100, 'temperature', -100, 60)

        assert is_valid is False
        assert 'above maximum' in error

class TestDataModels:
    """Unit tests for data model construction."""

    def test_location_creation(self):
        """Test creating valid Location object."""
        location = Location(
            name="London",
            country="GB",
            state="England",
            latitude=51.5074,
            longitude=-0.1278,
            confidence_score=0.95,
            raw_data={}
        )

        assert location.name == "London"
        assert location.latitude == 51.5074
        assert location.confidence_score == 0.95

    def test_weather_condition_creation(self):
        """Test creating valid CurrentWeather object."""
        weather = CurrentWeather(
            temperature=15.3,
            feels_like=13.8,
            humidity=72,
            pressure=1013.2,
            description="Light Rain",
            icon="10d"
        )

        assert weather.temperature == 15.3
        assert weather.description == "Light Rain"

    def test_weather_data_creation(self):
        """Test creating complete WeatherData object."""
        location = Location(
            name="London",
            country="GB",
            state=None,
            latitude=51.5074,
            longitude=-0.1278,
            confidence_score=0.95,
            raw_data={}
        )

        current = CurrentWeather(
            temperature=15.3,
            feels_like=13.8,
            humidity=72,
            pressure=1013.2,
            description="Clear",
            icon="01d"
        )

        weather_data = WeatherData(
            location=location,
            current=current,
            timestamp=datetime.now(),
            timezone_offset_seconds=3600,
            data_quality_score=0.92,
            validation_warnings=[],
            raw_data={}
        )

        assert weather_data.location.name == "London"
        assert weather_data.current.temperature == 15.3
        assert weather_data.data_quality_score == 0.92

# Run unit tests with: pytest tests/test_components.py -v
Terminal 
tests/test_components.py::TestInputValidator::test_valid_city_name PASSED
tests/test_components.py::TestInputValidator::test_city_with_state PASSED
tests/test_components.py::TestInputValidator::test_coordinates PASSED
tests/test_components.py::TestInputValidator::test_empty_input PASSED
tests/test_components.py::TestInputValidator::test_input_too_short PASSED
tests/test_components.py::TestInputValidator::test_input_too_long PASSED
tests/test_components.py::TestInputValidator::test_whitespace_normalization PASSED
tests/test_components.py::TestErrorCategorization::test_empty_input_categorized_as_user_input PASSED
tests/test_components.py::TestErrorCategorization::test_too_long_input_categorized_as_user_input PASSED
tests/test_components.py::TestErrorCategorization::test_network_timeout_categorized_as_transient PASSED
tests/test_components.py::TestErrorCategorization::test_connection_error_categorized_as_transient PASSED
tests/test_components.py::TestErrorCategorization::test_keyerror_categorized_as_not_found PASSED
tests/test_components.py::TestDataValidation::test_validate_structure_success PASSED
tests/test_components.py::TestDataValidation::test_validate_structure_missing_section PASSED
tests/test_components.py::TestDataValidation::test_validate_range_within_bounds PASSED
tests/test_components.py::TestDataValidation::test_validate_range_below_minimum PASSED
tests/test_components.py::TestDataValidation::test_validate_range_above_maximum PASSED
tests/test_components.py::TestDataModels::test_location_creation PASSED
tests/test_components.py::TestDataModels::test_weather_condition_creation PASSED
tests/test_components.py::TestDataModels::test_weather_data_creation PASSED

==================== 20 passed in 0.12s ====================

Twenty tests, 0.12 seconds. That speed is what makes the test pyramid economical: this loop is cheap enough to run on every save and still catches the silent regressions that creep into individual functions. If you see fewer than twenty PASSED lines, the most likely cause is ModuleNotFoundError: pytest finds the test file but can't import one of its dependencies. Check that you're running from the project root and that every module the tests import is saved alongside the tests/ folder.

Integration tests apply the same shape one level up: same pyramid, same pytest invocation, but the unit under test is now the seam between two layers.

tests/test_integration.py: contracts between layers, network mocked

Same shape, broader scope. Each test exercises a hand-off: does the API client return the dataclass shape the processor expects? Does the orchestrator package a partial-success result the way the display layer can render? Save the file at tests/test_integration.py and run it with pytest tests/test_integration.py:

tests/test_integration.py 
"""
Integration tests for weather dashboard.

Tests layer interactions and data flow through the system.
Uses mocking to avoid external API dependencies.
"""
import pytest
from unittest.mock import Mock, patch, MagicMock
from datetime import datetime

from weather_api_client import WeatherAPIClient, APIResult
from geocoding_processor import GeocodingProcessor
from weather_processor import WeatherProcessor
from orchestrator import WeatherOrchestrator

class TestAPIClientIntegration:
    """Test API client with retry logic."""

    @patch('requests.get')
    def test_successful_request(self, mock_get):
        """Test successful API request."""
        # Mock successful response
        mock_response = Mock()
        mock_response.status_code = 200
        mock_response.json.return_value = [{'name': 'London'}]
        mock_get.return_value = mock_response

        client = WeatherAPIClient(api_key="test_key")
        result = client.fetch_geocoding("London")

        assert result.success is True
        assert result.data is not None
        assert isinstance(result.data, list)
        assert result.data[0]['name'] == 'London'

    @patch('requests.get')
    @patch('time.sleep')
    def test_retry_on_timeout(self, mock_sleep, mock_get):
        """Test that client retries on timeout."""
        import requests

        # Patch time.sleep so the retry backoff is instant, not the real
        # one-plus-second waits, keeping this test at unit-test speed.

        # First two calls timeout, third succeeds
        mock_response = Mock()
        mock_response.status_code = 200
        mock_response.json.return_value = []

        mock_get.side_effect = [
            requests.exceptions.Timeout(),
            requests.exceptions.Timeout(),
            mock_response
        ]

        client = WeatherAPIClient(api_key="test_key")
        result = client.fetch_geocoding("London")

        # Should have tried 3 times
        assert mock_get.call_count == 3
        assert result.success is True

    @patch('requests.get')
    def test_no_retry_on_404(self, mock_get):
        """Test that client doesn't retry 404 errors."""
        import requests

        mock_response = Mock()
        mock_response.status_code = 404

        error = requests.exceptions.HTTPError()
        error.response = mock_response
        mock_get.side_effect = error

        client = WeatherAPIClient(api_key="test_key")
        result = client.fetch_geocoding("NonexistentCity")

        # Should only try once (no retry for 404)
        assert mock_get.call_count == 1
        assert result.success is False

class TestProcessorIntegration:
    """Test processor with API client results."""

    def test_geocoding_processor_with_valid_response(self):
        """Test geocoding processor with valid API response."""
        # Create mock API result
        api_response = APIResult(
            success=True,
            data=[{
                'name': 'London',
                'country': 'GB',
                'state': 'England',
                'lat': 51.5074,
                'lon': -0.1278
            }],
            error_context=None
        )

        processor = GeocodingProcessor()
        locations = processor.process_response(api_response.data, "London")

        assert len(locations) > 0
        assert locations[0].name == "London"
        assert locations[0].latitude == 51.5074

    def test_geocoding_processor_with_empty_results(self):
        """Test processor handles empty results gracefully."""
        api_response = APIResult(
            success=True,
            data=[],
            error_context=None
        )

        processor = GeocodingProcessor()

        with pytest.raises(Exception) as exc_info:
            processor.process_response(api_response.data, "Xqzthlptv99999")

        assert "No locations found" in str(exc_info.value)

    def test_weather_processor_with_valid_response(self):
        """Test weather processor with valid API response."""
        from models import Location

        location = Location(
            name="London",
            country="GB",
            state=None,
            latitude=51.5074,
            longitude=-0.1278,
            confidence_score=0.95,
            raw_data={}
        )

        api_response = APIResult(
            success=True,
            data={
                'main': {
                    'temp': 15.3,
                    'feels_like': 13.8,
                    'humidity': 72,
                    'pressure': 1013.2
                },
                'weather': [{
                    'description': 'light rain',
                    'icon': '10d'
                }],
                'dt': int(datetime.now().timestamp()),
                'timezone': 3600
            },
            error_context=None
        )

        processor = WeatherProcessor()
        weather_data = processor.process_response(api_response.data, location)

        assert weather_data.location.name == "London"
        assert weather_data.current.temperature == 15.3
        assert weather_data.data_quality_score > 0

class TestOrchestratorIntegration:
    """Test complete workflow orchestration."""

    @patch('weather_api_client.WeatherAPIClient.fetch_geocoding')
    @patch('weather_api_client.WeatherAPIClient.fetch_weather')
    def test_successful_workflow(self, mock_weather, mock_geocoding):
        """Test complete successful workflow."""
        # Mock geocoding response
        mock_geocoding.return_value = APIResult(
            success=True,
            data=[{
                'name': 'London',
                'country': 'GB',
                'state': 'England',
                'lat': 51.5074,
                'lon': -0.1278
            }],
            error_context=None
        )

        # Mock weather response
        mock_weather.return_value = APIResult(
            success=True,
            data={
                'main': {
                    'temp': 15.3,
                    'feels_like': 13.8,
                    'humidity': 72,
                    'pressure': 1013.2
                },
                'weather': [{
                    'description': 'clear sky',
                    'icon': '01d'
                }],
                'dt': int(datetime.now().timestamp()),
                'timezone': 3600
            },
            error_context=None
        )

        orchestrator = WeatherOrchestrator(api_key="test_key")
        result = orchestrator.get_weather_for_city("London")

        assert result.succeeded
        assert result.has_weather
        assert result.weather_data.location.name == "London"

    @patch('weather_api_client.WeatherAPIClient.fetch_geocoding')
    def test_workflow_with_geocoding_failure(self, mock_geocoding):
        """Test workflow when geocoding fails."""
        mock_geocoding.return_value = APIResult(
            success=False,
            data=None,
            error_context={'category': 'not_found', 'detail': 'http_404'}
        )

        orchestrator = WeatherOrchestrator(api_key="test_key")
        result = orchestrator.get_weather_for_city("Xqzthlptv99999")

        assert not result.succeeded
        assert not result.has_weather
        assert result.error_message is not None

# Run integration tests with: pytest tests/test_integration.py -v

Mocking the network is what keeps these tests fast and deterministic. A real requests.get call would couple every test to upstream availability and make the suite flaky; the patched version verifies the layer contracts at unit-test speed. The real-network case still gets covered, but at the top of the pyramid where slow tests earn their place by being few.

tests/test_end_to_end.py: live API, sparse on purpose

Top of the pyramid, real network. These tests hit the OpenWeatherMap endpoints, walk a complete request from input to display, and confirm the full pipeline holds up against the live API. The pytestmark.skipif at the top of the file makes the whole suite a no-op when OPENWEATHER_API_KEY isn't set, so contributors without a key can still run the rest of the tests. Note that the suite reads the key directly from the environment, not via config.load_config(): in CI the key arrives through pipeline secrets rather than a local .env, so the test harness doesn't depend on the file's presence. Save the file at tests/test_end_to_end.py and run it with pytest tests/test_end_to_end.py:

tests/test_end_to_end.py 
"""
End-to-end tests for weather dashboard.

Tests complete workflows with real API calls.
Slower tests that verify production behavior.
"""
import pytest
import os
import time
from orchestrator import WeatherOrchestrator

# Skip E2E tests if no API key available
pytestmark = pytest.mark.skipif(
    not os.getenv('OPENWEATHER_API_KEY'),
    reason="OPENWEATHER_API_KEY not set"
)

class TestEndToEnd:
    """End-to-end tests with real APIs."""

    def setup_method(self):
        """Initialize orchestrator before each test."""
        api_key = os.getenv('OPENWEATHER_API_KEY')
        self.orchestrator = WeatherOrchestrator(api_key)

    def test_major_city_workflow(self):
        """Test complete workflow with major city."""
        result = self.orchestrator.get_weather_for_city("London")

        # Verify success
        assert result.succeeded
        assert result.has_weather

        # Verify location data
        assert result.weather_data.location.name
        assert -90 <= result.weather_data.location.latitude <= 90
        assert -180 <= result.weather_data.location.longitude <= 180

        # Verify weather data
        assert -100 <= result.weather_data.current.temperature <= 60
        assert 0 <= result.weather_data.current.humidity <= 100
        assert result.weather_data.current.description

        # Verify quality metadata
        assert 0 <= result.weather_data.data_quality_score <= 1
        assert result.weather_data.timestamp

    def test_city_with_country_workflow(self):
        """Test workflow with city and country specification."""
        result = self.orchestrator.get_weather_for_city("Paris, France")

        assert result.succeeded
        assert result.has_weather
        assert "Paris" in result.weather_data.location.name

    def test_nonexistent_city_workflow(self):
        """Test workflow with nonexistent city."""
        result = self.orchestrator.get_weather_for_city("Atlantis12345")

        # Should fail gracefully
        assert not result.succeeded
        assert not result.has_weather
        assert result.error_message is not None
        assert result.error_category is not None

    def test_empty_input_workflow(self):
        """Test workflow with empty input."""
        result = self.orchestrator.get_weather_for_city("")

        # Should fail at input validation
        assert not result.succeeded
        assert result.error_category == 'user_input'

    def test_multiple_requests_workflow(self):
        """Test that multiple requests work correctly."""
        cities = ["London", "Paris", "Tokyo"]
        results = []

        for city in cities:
            result = self.orchestrator.get_weather_for_city(city)
            results.append(result)

            # Small delay to avoid rate limiting
            time.sleep(1)

        # All should succeed
        assert all(r.succeeded for r in results)

        # Each should have different coordinates
        coords = [(r.weather_data.location.latitude,
                  r.weather_data.location.longitude) for r in results]
        assert len(coords) == len(set(coords)) # All unique

    def test_performance_benchmark(self):
        """Test that requests complete in reasonable time."""
        start_time = time.time()

        result = self.orchestrator.get_weather_for_city("London")

        duration = time.time() - start_time

        # Should complete in under 10 seconds
        assert duration < 10.0
        assert result.succeeded

Sparseness at this tier is economy, not laziness. Each E2E test consumes real API quota, takes seconds to minutes, and can fail because of upstream conditions you can't control. The six tests above are deliberately spread thin (happy path, major error categories, performance ceiling, multi-request sanity check), and that's the floor. Adding more usually means duplicating coverage that already lives in the integration tier, where the same scenario runs in milliseconds.

Coverage: 96% and where to look next

These four pytest invocations cover the workflow at every stage; the comments above describe what each one does. Run them from the project root:

Terminal 
# Run all tests with coverage report
pytest tests/ -v --cov=. --cov-report=term-missing

# Run only fast tests (unit + integration)
pytest tests/ -v --ignore=tests/test_end_to_end.py

# Run only end-to-end tests
pytest tests/test_end_to_end.py -v -s

# Run with performance timing
pytest tests/ -v --durations=10
Terminal 
==================== test session starts ====================
collected 34 items

tests/test_components.py::TestInputValidator::test_valid_city_name PASSED
tests/test_components.py::TestInputValidator::test_city_with_state PASSED
[... plus 18 more unit tests ...]

tests/test_integration.py::TestAPIClientIntegration::test_successful_request PASSED
tests/test_integration.py::TestAPIClientIntegration::test_retry_on_timeout PASSED
[... plus 6 more integration tests ...]

tests/test_end_to_end.py::TestEndToEnd::test_major_city_workflow PASSED
tests/test_end_to_end.py::TestEndToEnd::test_city_with_country_workflow PASSED
[... plus 4 more end-to-end tests ...]

==================== 34 passed in 15.34s ====================

---------- coverage: platform linux, python 3.11.5 -----------
Name                       Stmts   Miss  Cover
-----------------------------------------------
config.py                     28      0   100%
display.py                    94      2    98%
errors.py                     78      4    95%
geocoding_processor.py        89      3    97%
input_validator.py            67      2    97%
json_helpers.py               52      3    94%
models.py                     45      0   100%
orchestrator.py              156      7    96%
validators.py                 45      1    98%
weather_api_client.py        145      8    94%
weather_dashboard.py          68      1    99%
weather_processor.py         112      5    96%
-----------------------------------------------
TOTAL                        979     36    96%

Your specific numbers will differ slightly: which lines remain uncovered depends on which conditional branches your test data exercises, and per-module statement counts shift as you tune the code. The headline percentage and the per-module pattern are what stay stable across runs. 96% coverage on a four-layer system is plenty. 100% is rarely worth chasing: the missing 4% is usually defensive branches that fire only on conditions you can't reach from any test. What matters more than the headline number is which modules are showing gaps. Critical paths and layer boundaries should be green; gaps should land in the kind of if val is None: defensive scaffolding that you keep around precisely for the case the test can't reproduce.

Notice what the test suite doesn't have to do because the architecture did it first. The unit test for the error categoriser is one assertion line (assert category == 'user_input') because §4 made categorisation data, not an exception type the test would have to import-and-instance. The integration tests assert on weather_data.location.name == "London" because §5's processors return validated dataclasses, not nested dicts the test would have to walk. The E2E tests check result.succeeded and result.has_weather because §6's orchestrator folds every outcome into one shape, so the test never has to branch on what kind of failure occurred. Tests that look short are usually short because the system under them was designed to be testable; tests that look long are usually paying for an earlier shortcut.

With the pyramid in place, a change in any layer surfaces its damage at the cheapest tier that can catch it: a broken function in milliseconds, a broken layer hand-off in seconds, a broken pipeline before deploy. That's the lead's promise delivered, and it's the difference between a millisecond feedback loop and a user-visible regression.

Section 10 closes the chapter with a recap of what each layer earned and a quiz that pressure-tests the architectural calls before Part III opens with OAuth.