Imagine a system having N services, and all of them are loosely coupled. We know that because of the non-stop business needs, new features are added to the system continuously. Also, in real life, the transition of legacy applications to new architectures is not easy at all. Therefore, we must move each module piece by piece to microservices, leading to a constant deployment necessity. How can we know which part of the application these microservices will affect and whether the other services that consume these services will break or not?
This is where contract testing comes in handy.
Contract testing is a methodology to ensure that two different systems are compatible and can communicate. It captures the information exchanged between each service, storing them in a contract, which can be used to ensure that both sides satisfy it. Contract testing is not like schema testing; it requires both client and provider to agree on the interactions and allows for change over time.
What Is The Difference?
The difference between contract testing and other approaches is that its primary goal is to test each system independently from the others. The contract is generated by the code, meaning it is always kept up to date with reality.
-1.webp?width=320&height=239&name=pasted%20image%200%20(1)-1.webp)
The Advantages Of Contract Tests Over Integration Tests
Contract tests generally have the opposite properties to e2e integrated tests:
- Contract tests run fast because they don't need to communicate with various systems at a time.
- Contract tests are easier to maintain, so that you don't need to understand the whole domain to write tests.
- Contract tests are easy to debug and fix because the problem is only ever in the component you're testing - so you generally get a line number or a specific API endpoint that is failing.
- Contract tests uncover bugs locally: contract tests can and should run on developer machines before deploying the code.
From a business point of view, it is well known that the later a bug is found in a project lifecycle, the more costly it is to fix.
Contract Testing Keywords
Before going further, let’s get familiar with the keywords used for contract testing.
- A contract is a document that prescribes the expected API request, response, path, query parameters, headers, etc.
- A consumer is a side of a contract that consumes or uses a given API. It is also referred to as a client.
- A provider is a side of a contract that provides or simply owns the given API.
Provider-driven Contract Testing
In a provider-driven approach, provider drives the API evolution, publishing new contracts (and associated test doubles) whenever there’s a significant change.
The contracts’ main goal is to ensure that the provider implementation satisfies the request/response specification completely, regardless of whether his clients use all or just a tiny subset of the data in a response. In simple terms, the provider defines a “This is what I do and what I expect you to do” kind of contract for his clients, and it’s up to the consumers to select the one that is interesting.
Pros
The provider-driven approach to contract testing allows the provider to have complete control of the API development to decide the naming and versioning. This is useful if you are developing a provider service with a well-defined business domain because you will be the first one to know the new features and their effect on the API.
This approach is also convenient for a provider that releases a public API and can’t work closely with its consumers. It enables new clients to understand the capabilities of an API just by looking at the published stubs.
Cons
Provider-driven contract testing does not care so much about the consumer perspective. It doesn’t stop the provider from producing a hard-to-use API. It’s easy to misunderstand the expectations of consumers towards your API.
Another downside of not knowing how your consumers use your API is that when you want to modify it in a breaking way, you must assume that all your clients may need your complete response. Or, the hard way, ask around and double-check with each of your clients — if only you know them and have a chance to reach out to them.
Consumer-driven Contract Testing
In a consumer-driven approach to contract testing, the consumer drives the changes to the provider’s API.
The contracts are based on client integrations with the provider rather than specifying the whole request/response structure. For a provider, it’s a valuable perspective — each of its clients defines clearly which fields they use. Consumer defines a “This is what I exactly need” kind of contract for the provider to satisfy.
Does it mean the provider is no longer the authority of his API and has to follow all the requirements coming from all the consumers? Not necessarily. It does not mean that consumers have the right to decide on everything, including naming or data structures. Instead, a pull request can be opened in the consumer’s codebase, which consumers and providers may discuss. As a provider, it’s okay to disagree with the proposal and suggest consumers to make some changes. However, the important thing is the discussion between the consumer and provider. If you are the provider, you get to know your client’s needs and clearly see how your consumer will use your service’s API. You can also have an understanding of how your client perceives your domain.
Pros
Using the consumer-driven approach to contract tests results in well-designed APIs which are easier to use for the consumers. From the provider perspective, the feedback loop on the quality of your API definition is much shorter as it involves the consumers as early as at the design stage.
The provider also knows precisely which endpoints or fields from the response are actively used and required by its consumers. It Lowers maintenance costs, i.e., makes API/field deprecation easier — no more endless texts between you and your clients to find out if anyone uses the xyz field from your response. As a provider, you can simply run a set of tests against the contracts defined by your consumers and get the answer immediately.
Cons
What are the disadvantages then? First of all, there’s no single view of all the capabilities of an API — they vary between different clients’ contracts. This may put some additional effort into fully understanding the API’s picture on consumer teams that aim to start using it without any background as opposed to the provider-driven approach.
Regardless of your choice between provider-driven or consumer-driven contract tests, remember that provider and consumer should create contracts in close cooperation. Even the most comprehensive tools cannot take over the discussion between the members of two different teams. A well-defined contract that is up-to-date delivers a lot of value to both parties, which makes them both responsible for the contract, no matter whose codebase contains the code describing the contract.
Consumer-Driven Contract Testing using Pact Java
To demonstrate an example of Consumer-Driven Contracts, I prepared the following microservice application built using Spring Boot:
- Date Provider MicroService – /provider/validDate – Validates whether the given date is a valid date or not.
- Age Consumer MicroService – /age-calculate – Returns age of a person based on a given date.
Starting date-provider MicroService, which by default runs in port 8080:
mvn spring-boot:run -pl date-provider
Then start age-consumer MicroService, which by default runs in port 8081:
mvn spring-boot:run -pl age-consumer
Since the application is developed using Spring Boot in Java, I will use Pact for consumer-driven contract testing. Pact provides a DSL for defining contracts. In addition, Pact offers good integration with test frameworks such as JUnit, Spock, ScalaTest, as well as with build tools such as Maven and Gradle.
Let’s see examples of consumer and provider tests using Pact.
Consumer Testing
Consumer-Driven Contract testing begins with a consumer defining the contract. First of all, I have to add this dependency to my project:
au.com.dius
pact-jvm-consumer-junit5
4.0.9
test
Then add the below dependency to write java 8 lambda DSL to use with JUnit to build consumer tests. A Lamda DSL for Pact is an extension of the pact DSL provided by pact-jvm-consumer.
au.com.dius
pact-jvm-consumer-java8
4.0.9
test
Consumer tests start with creating requirements on the mock HTTP server. Let’s start with the stub:
@Pact(consumer = "ageConsumer")
public RequestResponsePact validDateFromProvider(PactDslWithProvider builder) {
Map<string, string=""> headers = new HashMap<string, string="">();
headers.put("content-type", "application/json");
return builder
.given("valid date received from provider")
.uponReceiving("valid date from provider")
.method("GET")
.queryMatchingDate("date", "1998-02-03")
.path("/provider/validDate")
.willRespondWith()
.headers(headers)
.status(200)
.body(LambdaDsl.newJsonBody((object) -> {
object.numberType("year", 1996);
object.numberType("month", 8);
object.numberType("day", 3);
object.booleanType("isValidDate", true);
}).build())
.toPact();
}
The above code is quite similar to what we do with API mocks using WireMock. We can define the input, which is HTTP GET method against the /provider/validDate path, and the output is the below JSON body:
{
"year": 1996,
"month": 8,
"day": 3,
"isValidDate": true
}
In the above lambda DSL, I used numberType and booleanType to generate a matcher that checks the type whereas numberValue and booleanValue specify a value in the contract. Using matchers reduces tight coupling between consumers and producers. Values like 1996, 8 are the dummy values returned by the mock server.
The next one is the test:
@Test
@PactTestFor(pactMethod = "validDateFromProvider")
public void testValidDateFromProvider(MockServer mockServer) throws IOException {
HttpResponse httpResponse = Request.Get(mockServer.getUrl() + "/provider/validDate?date=1998-02-03")
.execute().returnResponse();
assertThat(httpResponse.getStatusLine().getStatusCode()).isEqualTo(200);
assertThat(JsonPath.read(httpResponse.getEntity().getContent(), "$.isValidDate").toString()).isEqualTo("true");
}
@PactTestFor annotation connects the Pact method with a test case. The last thing I need to add before running the test is @ExtendWith and @PactTestFor annotation with the name of the provider.
@ExtendWith(PactConsumerTestExt.class)
@PactTestFor(providerName = "dateProvider", port = "1234")
public class PactAgeConsumerTest {
Maven command to execute the consumer test is:
mvn -Dtest=PactAgeConsumerTest test -pl age-consumer
The test will pass and the JSON file containing a contract will be generated in the target directory (target/pacts).
{
"provider": {
"name": "dateProvider"
},
"consumer": {
"name": "ageConsumer"
},
"interactions": [
{
"description": "valid date from provider",
"request": {
"method": "GET",
"path": "/provider/validDate",
"query": {
"date": [
"1998-02-03"
]
},
"matchingRules": {
"query": {
"date": {
"matchers": [
{
"match": "date",
"date": "1998-02-03"
}
],
"combine": "AND"
}
}
},
"generators": {
"body": {
"date": {
"type": "Date",
"format": "1998-02-03"
}
}
}
},
"response": {
"status": 200,
"headers": {
"content-type": "application/json",
"Content-Type": "application/json; charset=UTF-8"
},
"body": {
"month": 8,
"year": 1996,
"isValidDate": true,
"day": 3
},
"matchingRules": {
"body": {
"$.year": {
"matchers": [
{
"match": "number"
}
],
"combine": "AND"
},
"$.month": {
"matchers": [
{
"match": "number"
}
],
"combine": "AND"
},
"$.day": {
"matchers": [
{
"match": "number"
}
],
"combine": "AND"
},
"$.isValidDate": {
"matchers": [
{
"match": "type"
}
],
"combine": "AND"
}
},
"header": {
"Content-Type": {
"matchers": [
{
"match": "regex",
"regex": "application/json(;\\s?charset=[\\w\\-]+)?"
}
],
"combine": "AND"
}
}
}
},
"providerStates": [
{
"name": "valid date received from provider"
}
]
}
],
"metadata": {
"pactSpecification": {
"version": "3.0.0"
},
"pact-jvm": {
"version": "4.0.9"
}
}
}
Every interaction has a:
- Description
- Provider state – Allows the provider to set up a state.
- Request – Consumer makes a request.
- Response – Expected response from the provider.
Then the generated pact file is published to the pact broker by the consumer. Now it’s time for the producers to verify the contract messages shared via pact broker.
Verifying the Contract
In this case, the provider is a simple Spring boot application. First, I have to add the following dependency to my project:
<dependency>
<groupId>au.com.dius</groupId>
<artifactId>pact-jvm-provider-junit5</artifactId>
<version>4.0.10</version>
</dependency>
I need to define a way for the provider to access the pact file in a pact broker. @PactBroker annotation takes the hostname and port number of the actual pact broker URL. With the @SpringBootTest annotation, Spring Boot provides a convenient way to start up an application context used in a test.
@Provider("dateProvider")
@Consumer("ageConsumer")
@PactBroker(host = "localhost", port = "8282")
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class PactAgeProviderTest {
I also have to tell pact where it can expect the provider API.
@BeforeEach
void before(PactVerificationContext context) {
context.setTarget(new HttpTestTarget("localhost", port));
}
@LocalServerPort
private int port;
Then I will publish all the verification results back to the pact broker by setting the environment variable.
@BeforeAll
static void enablePublishingPact() {
System.setProperty("pact.verifier.publishResults", "true");
}
We will inform the JUnit about how to perform the test as follows:
@TestTemplate
@ExtendWith(PactVerificationInvocationContextProvider.class)
void pactVerificationTestTemplate(PactVerificationContext context) {
context.verifyInteraction();
}
@State annotation in the method is used for setting up the system to the state expected by the contract.
@State("valid date received from provider")
public void validDateProvider() {
}
Maven command to execute the provider test is:
mvn -Dtest=PactAgeProviderTest test -pl date-provider
-
Any changes made by the provider, like adding a new field or removing an unused field in the contract, will not break consumers’ build as they care only about the parameters or attributes in the existing contract.
-
Any changes made by the provider, like removing a used field or renaming it in the contract, will violate the contract and break consumers’ build.
-
Adding a new interaction by the consumer generates a new pact file, and the same needs to be verified by the provider in the pact broker as well.
There is also a more detailed PoC for contract testing using Pact that we implemented. Feel free to check it out!
Contract Testing using Pact