Editor’s Note: In this series of blogs, we’ve chosen a series of software development design patterns to help developers keep up to date on the patterns most important to the DevOps industry. Previous patterns include value stream management and anti-corruption layer.
Name
Customer Supplier Pipeline
Intent
In the microservices world, teams design independently deployable artifacts surrounding single business capabilities. These teams control their destiny by setting their own release cadence. Others perform contract testing between supplier artifacts and customer artifacts to ensure customers can consume what suppliers produce without engaging in intense integration tests between customers and suppliers.
Let’s consider an example to understand the relationship between customers and suppliers. A supplier could be a Java API that is consumed by both web and mobile applications like iOS, Android and Node.js. The Java API, although a supplier for the web and mobile apps, may be consuming data from an Oracle database, which supplies persistence. To provide a holistic product experience, these teams need to inter-operate without getting tied at their hips.
Let’s consider the following scenarios depicting how typical iOS, Android, Node.js, Java and Oracle products are released.
Scenario 1
The iOS, Android and Node.js teams wait for the Java API to be delivered so that they can perform integration tests. Serial execution of this nature has a cost of delay and inflates cycle time.
The “Customer Supplier Pipeline” design pattern advocates customers of the Java API to proceed with test doubles and contract tests without waiting for its finalized version.
Scenario 2
The Java API is unable to complete its work in a timely fashion since the contractual requirements with the front-end teams are unclear. Also, who should drive the contract – the Java API or the iOS, Android and Node.js teams?
The “Customer Supplier Pipeline” design pattern insists that contractual agreements should be a business decision and should not be guided by the opinions of individual teams. This pattern advocates that customers drive the contract and that suppliers honor the contract before going to production. This is contrary to a related pattern “Conformist Pipeline” (discussed later under “Related Patterns”) where a supplier can break its customers’ interfaces and still go to production. In the latter case, the onus lies on the customers to make adjustments.
Also known as
Producer Consumer Pipeline
Motivation
Continuous everything (integration | testing | delivery | deployment | analytics | governance) enables the iOS, Android, Node.js, Java and Oracle teams to release high-quality and secure software frequently and predictably in small increments. Without contract testing, the only way to know that services can communicate is by using expensive and brittle integration tests. Integration testing is one of the most dreaded steps in continuous everything pipelines, not just due to the long duration of tests but also due to the complexity involved in setting up and maintaining integration environments.
Let’s review the primary motives behind contract tests and the “Customer Supplier Pipeline” design pattern. As illustrated in the figure, the supplier pipeline should build code, pass tests, pass static code analysis checks, publish versioned artifacts of the supplier service and its double. The customer pipeline should build code, pass unit tests, pass static code analysis checks, publish versioned artifacts of the customer service, its double and the customer-driven contracts. Both the customer and supplier pipelines execute in parallel to this point, after which they join, or fan-in, to:
Deploy the supplier double, the customer double and the customer-driven contracts
Execute contract and other tests
Go live in production!
The bottom line is that the “Customer Supplier Pipeline” design pattern disentangles inter-twined customer and supplier pipelines so that they release customer and supplier versioned artifacts systematically.
Applicability
In a typical continuous everything pipeline, there are several instances of upstream (i.e. supplier) and downstream (i.e. customer) priorities factoring into each other. We have noted, time and again, that continuous everything principles apply not just to software modules. As with many other design patterns, the “Customer Supplier Pipeline” design pattern and its contractual agreements apply to firmware, embedded systems, hardware and IoT (internet of things).
From an architectural point of view, microservices artifacts can be designed around business capabilities and are independently deployable. Hence, the “Customer Supplier Pipeline” design pattern may not be as prevalent in the microservices world. On the contrary, in the service-oriented architecture (SOA) world, upstream suppliers can generate artifacts that are leveraged by downstream customers and this pattern has many applications.
With regard to who drives contracts in contract testing, there’s at least one crucial difference between the intranet and the internet worlds.
Intranet World
The “Customer Supplier Pipeline” design pattern makes more sense in the intranet world, where the supplier is aware of its customer(s) and can honor contracts written by customers before it goes live. This decision is ruled by business facts and does not rely on opinions made by individuals or teams.
Internet World
In the internet world, customer-driven contracts aren’t feasible. For example, we wouldn’t know who uses APIs offered by Google and Facebook, and how. Onus lies on the customers to respect the interfaces defined by Google and Facebook. The “Conformist Pipeline” design pattern, discussed under Related Patterns, makes more sense in the internet world, where the supplier is unaware of all its customer(s) and hence needs to drive the contract.
Consequences
The “Customer Supplier Pipeline” design pattern has rich benefits for teams looking to avoid crippling integration tests. However, it has some drawbacks.
Collaboration, Not Chaos
In a typical organization, there is no single person who has the entire context mapped between all participating suppliers and customers. The absence of an effective scrum of scrums leaves teams dwindling till the last minute in terms of requirements, timelines and quality. This leads to unproductive and intense bouts of hyper-collaboration, like a flurry of meetings that choke the calendar, calls that typically last longer than scheduled and emails flying around.
Remember, collaboration is only effective till a certain inflection point after which it results in chaos.
Death By A Thousand Contracts
Customer supplier teams experience delays in execution if contracts cannot be agreed upon between suppliers and customers. This pattern is a typical case of customer-driven contracts, whereby the customer gets to finalize the terms of the contract. Worse still, there could be multiple customers, who propose different contractual agreements, and alignment could take time.
Remember To Bring Your Own Double
Yes, BYOD. The supplier owns packaging and distribution of its own double, and so does the customer. It is dangerous for the customer to manufacture a supplier’s double, since the customer may be unfamiliar with the supplier’s latest enhancements, and vice versa. This could lead to false positives or false negatives, thereby leading teams to question the integrity of the pipeline.
Implementation
Customer-driven contract tests are at the heart of the “Customer Supplier Pipeline” design pattern. Contract testing is a way to ensure that services, such as an API provider and a client, can communicate with each other. Here’s a slick implementation of contract tests using Pact, a contract testing tool .
Known uses
The “Customer Supplier Pipeline” design pattern is used generously by most organizations, large and small, to perform contract testing in the intranet world.
Related patterns
There are a couple of closely-related patterns that we have already touched upon in the previous sections.
Conformist Pipeline
In contrast to the “Customer Supplier Pipeline” design pattern, the “Conformist Pipeline” design pattern allows the supplier to break its customers’ interfaces and still go to production. This is prevalent in the internet world, where the supplier may be unaware of its customers and hence the onus lies on the customers to make adjustments. See the figure for an illustration.
Supplier-driven Contract Testing
Contrary to the “Customer Supplier Pipeline” pattern, the “Conformist Pipeline” design pattern requires suppliers to set the terms of the contract and for customers to follow. This pattern is known as supplier-driven contract testing.
References
This DDD (domain-driven design) “customer supplier” pattern was first described by Eric Evans in his book “Domain-Driven Design: Tackling Complexity in the Heart of Software.”
Other than the DDD book, there are several articles that shed light on microservices, contract tests in general and how customer-driven contracts are used.
Additional resources
View the first post in this series on value stream management
Learn from the post on anti-corruption layers
Find out more about design patterns at DevOps World | Jenkins World 2019