Contract Testing with API Stubs in Pytest

In modern software development, applications often rely on external APIs or microservices to function. Ensuring that these integrations work correctly while maintaining flexibility requires contract testing. Contract testing validates that API consumers and providers adhere to predefined agreements, preventing breaking changes in distributed systems.

In this blog, we’ll cover:

• What contract testing is and why it’s essential
• Differences between contract testing, API stubbing, and mocking
• How to implement contract testing using Pact and OpenAPI Schema Validation
• Best practices for contract testing
• Useful GitHub repositories and AI-powered tools

🚀 What is Contract Testing?

Contract testing ensures that the interaction between a service consumer (client) and provider (API server) adheres to a defined contract.

For example:

• A frontend app expects an API to return {"id": 1, "name": "John Doe"} when requesting /users/1.
• If the backend suddenly changes the response format, it might break the frontend.
• Contract testing prevents such breaking changes by ensuring both sides agree on the expected API structure.

🆚 Contract Testing vs. API Stubbing vs. Mocking

✅ Implementing Contract Testing in Pytest

1️⃣ Approach 1: Using OpenAPI Schema Validation

If your API follows OpenAPI (Swagger) specifications, you can validate responses against the OpenAPI contract.

🔹 How It Works

• The test calls get_user(123), which fetches data from the API.
• validate_response(response) checks if the response matches the OpenAPI schema.
• If the API response does not match the schema, the test fails.

✅ Best For:

• Validating real API responses against OpenAPI contracts.
• Ensuring API changes do not break consumers.

# pip install pytest-openapi

# Consider an API with this OpenAPI Specification (openapi.json)

{
  "openapi": "3.0.0",
  "info": {
    "title": "User API",
    "version": "1.0.0"
  },
  "paths": {
    "/users/{user_id}": {
      "get": {
        "summary": "Fetch user details",
        "responses": {
          "200": {
            "description": "User found",
            "content": {
              "application/json": {
                "schema": {
                  "type": "object",
                  "properties": {
                    "id": {"type": "integer"},
                    "name": {"type": "string"},
                    "email": {"type": "string"}
                  },
                  "required": ["id", "name", "email"]
                }
              }
            }
          }
        }
      }
    }
  }
}

# Now, let’s write a contract test using pytest-openapi.

import requests
import pytest
from pytest_openapi import validate_response

def get_user(user_id):
    """Function to fetch user data from API"""
    return requests.get(f"https://api.example.com/users/{user_id}")

@pytest.mark.openapi("openapi.json")
def test_user_api_contract():
    """Contract test to validate API response against OpenAPI schema"""
    response = get_user(123)
    validate_response(response)

🔥 2️⃣ Approach 2: Consumer-Driven Contract Testing with Pact

Pact is a consumer-driven contract testing tool that ensures API consumers and providers stay in sync.

How Pact Works

• The consumer (client) defines expectations for the API.
• A Pact contract is generated based on these expectations.
• The provider (server) runs tests against the contract to ensure compliance.

🔹 How It Works

• The consumer generates a Pact file containing the expected API response.
• The provider runs tests against this contract to confirm that the API still behaves as expected.

✅ Best For:

• Microservices & distributed systems
• APIs used by multiple consumers

Step 1: Define the Consumer Contract (Stub API Response)

# Install Pact for Python
# pip install pact-python pytest

from pact import Consumer, Provider
import requests

pact = Consumer("UserConsumer").has_pact_with(Provider("UserProvider"))

def get_user_from_api(user_id):
    return requests.get(f"http://localhost:5000/users/{user_id}")

def test_consumer_pact():
    expected_response = {"id": 123, "name": "John Doe", "email": "john@example.com"}
    
    with pact:
        pact.given("User 123 exists").upon_receiving("A request for user 123").with_request(
            method="GET", path="/users/123"
        ).will_respond_with(200, body=expected_response)

        response = get_user_from_api(123)
        assert response.json() == expected_response

Step 2: Verify the Contract on the Provider Side

# The API provider should run the following to verify the contract:

from pact.verify import ProviderVerifier

def test_provider_pact():
    verifier = ProviderVerifier()
    assert verifier.verify_pacts(pact_dir="./pacts", provider_base_url="http://localhost:5000")

🎯 Handling Edge Cases in Contract Testing

1️⃣ Testing for Missing Fields

If the API removes a required field, the contract test should fail.

@pytest.mark.openapi("openapi.json")
def test_missing_email():
    response = get_user(123)
    data = response.json()
    assert "email" in data, "Email field is missing in API response"

2️⃣ Validating Response Formats

# Ensure that fields have the correct data types.

@pytest.mark.openapi("openapi.json")
def test_user_id_is_integer():
    response = get_user(123)
    assert isinstance(response.json()["id"], int)

3️⃣ Checking Backward Compatibility

# If an API provider introduces new optional fields, the contract should still pass.

@responses.activate
def test_new_optional_field():
    responses.add(
        responses.GET, 
        "https://api.example.com/users/123",
        json={"id": 123, "name": "John Doe", "email": "john@example.com", "age": 30}, 
        status=200
    )
    
    response = get_user(123)
    assert "age" in response.json()

🏆 Best Practices for Contract Testing

✅ Keep contracts version-controlled – Store OpenAPI or Pact contracts in Git repositories.
✅ Test API schema changes in CI/CD pipelines – Break builds if contracts fail.
✅ Validate both request and response structures – Ensure that API clients send valid requests too.
✅ Run contract tests before deploying API changes – Prevent breaking production consumers.
✅ Use pytest-openapi for schema validation and Pact for consumer-driven contracts.

📚 Further Reading & Resources

🔗 Popular Resources

• Pact Contract Testing Docs
• OpenAPI Specification Guide
• pytest-openapi

🎯 Underrated Resources

• Contract Testing in CI/CD
• Using Pact for API Evolution

📌 GitHub Repositories

• pact-python
• pytest-openapi

🤖 AI Tools for Contract Testing

• Postman Contract Tests – Validate API contracts with test scripts
• Swagger Codegen AI – Automate API contract generation

🎯 Conclusion

Contract testing ensures APIs remain stable and compatible across updates. Using OpenAPI validation and Pact, developers can automate API schema verification and prevent breaking changes.

Would you like a deep dive into API versioning strategies next? 🚀