The debate often gets framed as a choice: consumer-driven contracts or recorded stubs? Teams that have tried both sometimes treat them as competing philosophies: one formally correct, the other a practical workaround. The framing is wrong. They answer different questions, catch different failures, and compose well. Choosing one doesn’t preclude the other.
What contract testing actually verifies
Contract testing, implemented in tools like Pact, Spring Cloud Contract, and bi-directional contract testing (BDCT), addresses a specific coordination problem: two teams, a consumer and a provider, need to agree on the shape of an interaction and detect when someone breaks the agreement.
In Pact’s model, the consumer generates a pact file: a JSON document describing the interactions it expects: the request it will send and the minimum response it needs. The pact is published to a broker (Pact Broker or PactFlow). The provider then runs provider verification: it replays each interaction against its running service and confirms it can satisfy every consumer expectation. If the provider renames a field the consumer depends on, verification fails before that change merges.
Spring Cloud Contract inverts the authority: the provider owns the contract, defined as a Groovy DSL or YAML file. The framework generates both server-side tests for the provider and client-side stubs for consumers. The contract lives with the provider; consumers test against generated stubs.
Bi-directional contract testing (as implemented in PactFlow) takes a third approach: the consumer’s contract is derived from its test suite, and the provider’s contract is derived from an OpenAPI spec or a recorded response set. Compatibility is checked statically by comparing both representations, so no provider needs to run during consumer CI.
All three share a common property: what’s being tested is an agreed expectation. The consumer describes what it believes the API returns. The provider confirms it can deliver that. If both parties never discussed a particular field, it isn’t in the contract, and a change to that field will not be caught.
This is not a deficiency; it’s the design. Contracts are a coordination mechanism. They’re precisely as comprehensive as the conversations that produced them.
What recorded stubs verify
Recorded stubs verify something different: how the API actually behaves, including behavior nobody explicitly decided on.
When you record real API traffic and replay it in tests, the fixture reflects what the API returned when traffic was captured, not what you understood it would return when you wrote the consumer. This means:
- Fields the upstream added without announcing them are in your fixture.
- Enum values that appear in production but aren’t in the docs show up in your test data.
- Edge cases like the
nullwhere the spec said “optional string”, the timestamp with a non-UTC offset, or the numeric ID that exceeds 32-bit range are captured at the rate they actually occur. - The response your code receives on the afternoon a third party quietly deploys a hotfix ends up in your fixture library the next time traffic is captured.
Recorded stubs don’t verify that an agreed contract is satisfied. They verify that your code handles what the API is actually doing. That’s a different question, and often a more uncomfortable one.
What each catches and what each misses
What contract testing catches: An intentional breaking change by the provider. If the provider team renames a field that a consumer depends on, provider verification fails before the change ships. Contract testing is most valuable for internal APIs between teams that can coordinate: it creates a machine-enforced dependency graph between services. The feedback arrives at the pull-request level, before anything reaches a shared environment.
What contract testing misses: The undocumented reality. A provider can satisfy all consumer contracts and still behave in ways consumers don’t handle: fields no consumer ever requested, edge-case values no contract enumerated, behavior at error paths nobody modeled. Consumer-driven contracts test the intersection of what was requested and what the provider delivers; they don’t test everything the provider actually delivers.
What recorded stubs catch: The undocumented reality. Stubs generated from real traffic include everything the API actually returned, not just what was formally modeled. A stub from production traffic will contain the extra field the upstream started returning two releases ago, the odd error shape that appears in 0.3% of requests, the response that triggers the obscure branch in your parser. These surface in the test suite before they surface in a production incident.
What recorded stubs miss: Intentional breaking changes that are coordinated through contracts. If the upstream provider removes a field that your recorded stubs happen to contain, your tests still pass, because you’re testing against what the API used to do, not what it does now. Re-recording is the remedy, but it requires deliberate action. Stale stubs are the primary failure mode of any recording-based system, and staying current requires a process, not just a tool.
A sane default split
A heuristic that holds up in practice:
Use contract testing for internal APIs you control on both sides. When your organisation owns the consumer and the provider, contract testing gives you a pull-request-level check that breaking changes don’t merge silently. The coordination overhead is worth it because you have the organisational leverage to enforce the workflow: both teams run CI that includes the contract step, and nobody ships a breaking change without seeing the failure first.
Use recorded stubs for third-party APIs and legacy edges you don’t control. You can’t run provider verification against a payment processor. You can’t ask a shipping carrier to sign off on a pact file. You can record what those APIs actually return and replay it in CI. When the third party changes their response format, your re-recording run captures it and surfaces the breakage in the test suite rather than in production.
Use both for large internal services at the boundary. For a central internal service that many consumers depend on, especially one with a rich and evolving response schema, contracts define the coordination layer and recorded stubs verify the actual runtime behavior. The contract tests catch negotiated changes; the recorded stubs catch everything the contract doesn’t cover.
Composing them in one CI pipeline
A pipeline that uses both looks roughly like this:
Stage 1: unit tests. Fast, no external dependencies, no stubs or contracts. Runs first.
Stage 2: recorded stub replay. Tests that exercise API client code run against stubs generated from real traffic. Coverage here is broad: any path through the client that production traffic has exercised. This stage runs against what the API has actually done, not what a contract says it should do.
Stage 3: contract verification. Triggered by the pact broker’s webhook on provider change, or as a required check on consumer PRs. This is the team-coordination step. It catches intentional changes that nobody informed the consumer team about.
Stage 4: integration smoke tests. Narrow and slow, run against a staging environment. Catches genuine environmental problems that neither stubs nor contracts can simulate.
Stages 2 and 3 cover different ground. Running both isn’t redundancy; it’s catching two distinct categories of failure at the cheapest point in the pipeline.
One practical note on ordering: when a provider team makes a change that breaks contracts, fix that first. Contract failures mean the provider changed something a consumer depends on; recorded stubs from before the change will still pass, which could give a false sense of safety. Run contract verification early enough that you’re not shipping to staging before catching that class of failure.
Closing thought
The “contracts vs. stubs” debate usually happens when a team has hit the limits of one approach and concluded that the alternative must be the real answer. The limits of one are not the strengths of the other; they have different shapes. Contract testing is a coordination protocol between teams. Recorded stubs are a fidelity mechanism against real behavior. A testing strategy that uses both isn’t hedging; it’s covering the failure modes that matter.
For teams evaluating recorded stubs from production-like traffic, with masking to keep sensitive data out of fixtures, see pricing for Stubsmith’s plans. A free tier is available for teams getting started.