Design Patterns in API Gateways and Microservices

For all the buzz about microservices and API gateways, finding specifics can prove surprisingly difficult. I am reminded of the cartoon by Sidney Harris where the first step of a complex mathematical formula is presented, then a miracle occurs, and the sudden appearance of the glorious solution prompts an observer to comment that perhaps we should be more explicit in step two.

Since these patterns solve problems that occur almost exclusively at scale, there is a distinct dearth of published articles that tackle some of the trickiest details of these implementations.

This article assumes that you are familiar with the benefits of microservices (smaller repositories, language-agnostic development, easier refactoring, et al) and that you understand the role of an API gateway as a facade in front of them. The goal of this article is to catalog some of the architectural patterns that come up as potential problems or solutions in the gateway + microservice landscape.

For those of you eyeing the gateway + microservice architecture as a source of potential relief from the compound problem of a monolithic application, we may have some bad news: the benefits of the gateway + microservice solution may have been overly simplified in its sales pitch. You may need to overcome some significant challenges and be a bit more explicit in "step two."

Even if your monolithic application is thoughtfully structured into packages and service classes, chances are good that there are certain aspects of your application's design that will make its components difficult to slice into dedicated microservices without refactoring.

Why? Because often, your application's components rely on functionality that is considered "global" in the scope of the application. That's what made it easier to develop in the first place.

Here is a short list of the most common cross-cutting concerns in applications:

• Authentication

• Authorization

• Sessions

• Cookies

• Cache

• Logging

• Dependencies on other services

Let's look more closely at each item.

Authentication in the gateway + microservice ecosystem is best handled by a service that produces either a JSON web token or some other auth token which can be included in subsequent requests. The token gets evaluated by the gateway (and only by the gateway) to determine whether a request is properly authenticated.

Closely related to authentication, authorization in the gateway + microservice ecosystem should be possible using a token (eg, sent in a custom HTTP header). This task should be performed before a request is proxied through to any microservice. Think of this as the single responsibility principle: each microservice only cares about one thing, and that thing cannot also include a permissions check.

As stated previously, the recommendation here is to avoid sessions in favor of tokens so you can avoid looking up user-specific data in your microservices. When needed, the gateway should pass session data (eg, from a decrypted token) along to the microservices.

Arguably, it is possible to pass only a session identifier and then let each microservice look up session data from the attached resource (eg, Redis) per the wise advice of the 12 Factor App.

That approach may make for easier refactoring in some cases, but not always. For example, PHP's session_start() and the resulting $_SESSION superglobal cannot be used in a microservice when the session ID is dictated by the gateway. In that case, you will need to roll your own solution. And if you are going to reinvent a wheel, it would be best to do it in a single place (in the gateway) and spare the microservices from that bit of tedious busywork.

Like sessions, cookies are best avoided by your microservices, and if needed, they are easier and cleaner to implement in the gateway. When absolutely required, microservices can emit cookies if the gateway is configured to proxy them, but then you may risk additional headaches trying to juggle the cookie domains.

There is no perfect solution for cache, so do not try to optimize your ecosystem prematurely. Ease into caching and start with small expiration times. Maintaining REST-friendly routes in your microservices will allow for simpler caching at higher levels (eg, Varnish).

Some use cases should consider the possibility of cached data being your model, ie, the source of truth. Event handlers and command-line services may help keep the cache updated.

Logging in a gateway + microservice ecosystem is best done using either a log aggregation service such as Loggly or by simply logging to stdout and then doing your own log aggregation.

A standardized logging format (eg, JSON with some required fields) is recommended. This will allow for consistent reporting across all components. Allow room in your log format for a request ID that can be passed from the gateway into each microservice so you can easily find log entries in any service that had a part in handling a specific request.

Code reuse in a monolithic application is usually a good thing, but reusing services in a microservice architecture may not be a good idea. Rethinking your code as standalone services takes time, and the refactoring may mean that clients need to make more or different requests.

A primary goal for the system is robustness: each microservice should be as independent as possible, and they should not risk cascading failures because one service outage triggers another.

!Sign up for a free Codeship Account

After covering the most common cross-cutting concerns that should be handled at the gateway level, we should have a much clearer idea of what the gateway needs to do.

It should act as a gated proxy, enforcing authorization rules so that only appropriate requests are passed through to each microservice. Just remember that the exact implementation details here can affect the microservices significantly.

One of the biggest decisions to be made is how the gateway will interface with its microservices. Will they be installed as packages or plugins (an approach used by frameworks such as Seneca), or will communication with microservices be conducted exclusively via HTTP (as used by Amazon API Gateway)? How you answer that question will affect the structure of your microservices.

After implementing several microservices, I started to observe certain tendencies in their structure. I'm hesitant to dub them "patterns," partly out of fear of angering the purists, but also because many of the recommendations here boil down to best practices that have been enumerated elsewhere.

Consideration of the following items will help you get the most out of a gateway + microservice implementation.

For educational purposes, it is helpful to start out with a simple example: let us consider a microservice that returns a country's full name when given its two-letter code. The data could be supplied by a single database table with no foreign keys required.

Per the advice of the 12 Factor App, we treat the database as a backing service and attach it as a resource. There are no other backing services required, only the database. We supply the database credentials in the environment.

An app this tiny is easy to set up, data migrations are trivial, and it is easy to test. This is a simple example of an "Application Model," sometimes referred to as Backends-as-frontends.

Next, let's consider something more substantial: a service that returns ferry travel times, but it checks against local weather conditions. Assuming the ferry routes are stored in a database, then we can attach it as a resource as we did in the previous example.

The weather information is read from a third-party API, so we must interface with it via an SDK or some sort of HTTP call. This example is trickier than the first because our service is dependent on another service. Since we don't control the other service, however, the best we can do is fail gracefully if the weather API goes down and hope for good uptime.

Sooner or later, you will find a use case in your application where you will consider the option to have one of your microservices depend on another. This approach is easy since services are developed as plugins, but it may have drawbacks when microservices are accessed via HTTP. Should that HTTP request go directly to the service, or should it go through the gateway to take advantage of its routes, auth, and versioning controls?

Regardless of how it's done, debugging becomes more difficult, and there is a distinct risk of cascading failures. What are the alternatives?

Consider the following alternatives if you find a situation where a dependency on another microservice seems tempting:

• Refactor the client to make an additional request to the other microservice.

• Simplify the microservice that was needed as a dependency. If you are trying to reuse it, then perhaps it has grown too complex. Microservices are perhaps the most valuable when they are so simple that there isn't a significant benefit from their reuse.

• Are you conflating a service with a model? If a microservice is dependent on another microservice because of data, then the one microservice should instead attach to the same data models as the other microservice, and the dependency can be avoided. If a microservice wants to reuse business logic, then you need to think about the best way to avoid repeating any of those rules. Perhaps the two microservices should be merged into one so the business logic can be reused without repetition.

• Is this something that a command-line service or cron job can help simplify? Judicious use of notifications and backend tasks can help keep microservices clean and focused.

Because "Application Models" may get convoluted (especially if they have multiple dependencies), some API gateways offer the possibility to aggregate data from different services, usually by adding code to the relevant route and/or controller. I have heard the idea of aggregating data in the gateway referred to as the "scatter-gather" approach.

The advantage to aggregating data in the gateway is that it spares the client from having to do arduous work assembling data from multiple requests. Likewise, it spares the microservices from having to juggle complex interactions.

This approach has a couple major drawbacks, however. If the thought of juggling multiple requests and applying business logic is undesirable in a client or in a microservice, it is probably even less so in the gateway. If your gateway performs aggregations, it will no longer be a simple proxy -- it will in effect include business logic that would need to be carefully tested.

Many available gateways do not offer this feature, so relying on it will limit your choices of gateways or force you to write your own, an unattractive proposition for businesses that derive value from their services and not from their custom gateways.

An alternative to aggregating data in the gateway is to aggregate data in the client. The most well-known implementation of this is perhaps GraphQL, but a thorough implementation of a JSON API service can accomplish something similar. The end result is that the onus of making and merging multiple data requests is left to the client; the gateway and its microservices are allowed to remain simple and streamlined.

The advantage to this approach is that it keeps the client in charge, so versioning is usually simpler (because releasing a new version of the client is usually easier than releasing a new version of the server-side API). This approach may also avoid the problems of under- and over-fetching data.

Although this approach may rely on relatively new technology, its biggest potential drawback is with business logic. If any business logic is required to interpret and merge multiple requests, then you risk repeating those rules in multiple clients and therefore you risk having inconsistent behavior.

Hopefully this discussion about the landscape of the gateway + microservice architecture has illuminated some of the questions and concerns that often accompany it.

There isn't a silver bullet solution that will meet everyone's needs, but there are certain trends that are picking up momentum. Technologies such as Serverless and AWS Lambda promise to make microservices even more granular, and it's hard to argue with the benefits of their simplicity and testability.

As you try out different solutions for different use cases following the guidelines in this article, keep your wits about you and beware of any solution that threatens to disrupt the simplicity and testability of your code. Just as with any architecture, repetition and practice with microservices will help you identify solutions that work.