Chapter 5. Challenges
The separation of a system into microservices makes the system as a whole more complex. This leads to challenges at the technical level (see section 5.1)—for instance, high latency times in the network or the failure of individual services. There are also a number of things to consider at the software architecture level—for instance, it can be difficult to move functionality between different microservices (section 5.2). Finally, there are many more components to be independently delivered—making operations and infrastructure more complex (section 5.3). These challenges have to be dealt with when introducing microservices. The measures described in the following chapters explain how to handle these challenges.
5.1 Technical Challenges
Microservices are distributed systems with calls between microservices going via the network. This can negatively impact both the response time and latency of microservices. The previously mentioned first rule for distributed objects1 states that objects, where possible, should not be distributed (see section 3.1).
1. http://martinfowler.com/bliki/FirstLaw.html
The reason for that is illustrated in Figure 5.1. A call has to go via the network to reach the server, is processed there, and has to then return to the caller. The latency just for network communication can be around 0.5 ms in a computing center (see here).2 Within this period of time a processor running at 3 Ghz can process about 1.5 million instructions. When computation is redistributed to another node, it should be checked to find out whether local processing of the request might be faster. Latency can further increase because of parameter marshaling and unmarshaling for the call and for the result of the call. Network optimizations or connecting nodes to the same network switch can improve the situation.
2. https://www.cs.cornell.edu/projects/ladis2009/talks/dean-keynote-ladis2009.pdf
The first rule for distributed objects and the warning to be aware of regarding the latency within the network dates back to the time when CORBA (Common Object Request Broker Architecture) and EJB (Enterprise JavaBeans) were used in the early two-thousands. These technologies were often used for distributed three-tier architectures (see Figure 5.2). For every client request the web tier only supplies the HTML for rendering the page. The logic resides on another server, which is called via the network. Data is stored in the database, and this is typically done on another server. When only data is to be shown, there is little happening in the middle tier. The data is not processed, just forwarded. For performance and latency reasons, it would be much better to keep the logic on the same server as the web tier. Although splitting the tiers between servers enables the independent scaling of the middle tier, the system does not get faster by doing this for situations where the middle tier has little to do.
For microservices the situation is different, as the UI is contained in the microservice. Calls between microservices only take place when microservices need the functionality offered by other microservices. If that is often the case, this might indicate that there are architectural problems, as microservices should be largely independent of each other.
In reality, microservice-based architectures function in spite3 of the challenges related to distribution. However, in order to improve performance and reduce latency, microservices should not communicate with each other too much.
3. http://martinfowler.com/articles/distributed-objects-microservices.html
Code Dependencies
A significant benefit of a microservice-based architecture is the ability to independently deploy the individual services. However, this benefit can be undone by code dependencies. If a library is used by several microservices and a new version of this library is supposed to be rolled out, a coordinated deployment of several microservices might be required—a situation that should be avoided. This scenario can easily occur because of binary dependencies where different versions are not compatible any more. The deployment has to be timed such that all microservices are rolled out in a certain time interval and in a defined order. The code dependency also has to be changed in all microservices, a process that has to be prioritized and coordinated across all the teams involved. A binary-level dependency is a very tight technical coupling, which leads to a very tight organizational coupling.
Therefore, microservices should adhere to a “shared nothing” approach where microservices do not possess shared code. Microservices should instead accept code redundancy and resist the urge to reuse code in order to avoid a close organizational link.
Code dependencies can be acceptable in certain situations. For instance, when a microservice offers a client library that supports callers using the microservice, this does not necessarily have negative consequences. The library will depend on the interface of the microservice. If the interface is changed in a backward-compatible manner, a caller having an old version of the client library can still use the micro-service. The deployment remains uncoupled. However, the client library can be the starting point to a code dependency. For instance, if the client library contains domain objects, this can be a problem. In fact, if the client library contains the same code for the domain objects that is also used internally, then changes to the internal model will affect the clients. This might mean they have to be deployed again. If the domain object contains logic, this logic can only be modified when all clients are likewise deployed anew. This also violates the principle of independently deployable microservices.
Unreliable Communication
Communication between microservices occurs over the network and is therefore unreliable. Additionally, individual microservices can fail. To ensure that a microservice failure does not lead to a failure of the entire system, the remaining microservices must compensate for the failure and enable the system to continue. However, to achieve this goal, the quality of the services may have to be degraded, for example, by using default or cached values or limiting the usable functionality (section 9.5).
This problem cannot be completely solved on a technical level. For instance, the availability of a microservice can be improved by using hardware with high availability. However, this increases costs and is not a complete solution; in some respects, it can even increase risk. If the microservice fails despite highly available hardware and the failure propagates across the entire system, a complete failure occurs. Therefore, the microservices should still compensate for the failure of the highly available microservice.
In addition, the threshold between a technical and a domain problem is crossed. Take an automated teller machine (ATM) as an example: When the ATM cannot retrieve a customer’s account balance, there are two ways to handle the situation. The ATM could refuse the withdrawal. Although this is a safe option, it will annoy the customer and reduce revenue. Alternatively, the ATM could hand out the money—maybe up to a certain upper limit. Which option should be implemented is a business decision. Somebody has to decide whether it is preferable to play it safe, even if it means foregoing some revenue and annoying customers, or to run a certain risk and possibly pay out too much money.
Technology Pluralism
The technology freedom of microservices can result in a project using many different technologies. The microservices do not need to have shared technology; however, the lack of common technology can lead to increasingly complexity in the system as a whole. Each team masters the technologies that are used in their own microservice. However, the large number of technologies and approaches used can cause the system to reach a level of complexity such that no individual developer or team can understand all of it any more. Often such a general understanding is not necessary since each team only needs to understand its own microservice. However, when it becomes necessary to look at the entire system—for example, from a certain limited perspective such as operations—this complexity might pose a problem. In this situation, unification can be a sensible countermeasure. This does not mean that the technology stack has to be completely uniform but that certain parts should be uniform or that the individual microservices should behave in a uniform manner. For instance, a uniform logging framework might be defined. The alternative is to define just a uniform format for logging. Then different logging frameworks could be used that implement the uniform format differently. Also a common technical basis like the JVM (Java Virtual Machine) can be decided upon for operational reasons without setting the programming languages.
5.2 Architecture
The architecture of a microservice-based system divides the domain-based pieces of functionality among the microservices. To understand the architecture at this level, dependencies and communication relationships between the microservices have to be known. Analyzing communication relationships is difficult. For large deployment monoliths there are tools that read source code or even executables and can generate diagrams visualizing modules and relationships. This makes it possible to verify the implemented architecture, adjust it towards the planned architecture, and follow the evolution of the architecture over time. Such overviews are central for architectural work; however, they are difficult to generate when using microservices as the respective tools are lacking—but there are solutions. Section 7.2 discusses these in detail.
Architecture = Organization
A key concept that microservices are based on is that organization and architecture are the same. Microservices exploit this situation to implement the architecture. The organization is structured in a way that makes the architecture implementation easy. However, the downside of this is that an architecture refactoring can require changes to the organization. This makes architectural changes more difficult. This is not only a problem of microservices; Conway’s Law (section 3.2) applies to all projects. However, other projects are often not aware of the law and its implications. Therefore, they do not use the law productively and cannot estimate the organizational problems caused by architectural changes.
Architecture and Requirements
The architecture also influences the independent development of individual micro-services and the independent streams of stories. When the domain-based distribution of microservices is not optimal, requirements might influence more than one microservice and therefore more than one team. This increases the coordination required between the different teams and microservices. This negatively influences productivity and undoes one of the primary reasons for the introduction of microservices.
With microservices the architecture influences not only software quality, but also the organization and the ability of teams to work independently and therefore productivity. Designing an optimal architecture is even more important since mistakes have far-reaching consequences.
Many projects do not pay sufficient attention to domain architecture, often much less than they pay to technical architecture. Most architects are not as experienced with domain architecture as with technical architecture. This situation can cause significant problems in the implementation of microservice-based approaches. The splitting of functionality into different microservices and therefore into the areas of responsibility for the different teams has to be performed according to domain criteria.
Refactoring
Refactoring a single microservice is straightforward since microservices are small. They can also be easily replaced and reimplemented.
Between microservices the situation is different. Transferring functionality from one microservice to another is difficult. The functionality has to be moved into a different deployment unit. This is always more difficult than moving functionality within the same unit. Technologies can be different between different microservices. Microservices can use different libraries and even different programming languages. In such cases, the functionality must be newly implemented in the technology of the other microservice and subsequently transferred into this microservice. However, this is far more complex than moving code within a microservice.
Agile Architecture
Microservices enable new product features to be rapidly delivered to end users and for development teams to reach a sustainable development speed. This is a particular benefit when there are numerous and hard-to-predict requirements. This is exactly the environment where microservices are at home. Changes to a microservice are also very simple. However, adjusting the architecture of the system, for instance, by moving around functionality, is not so simple.
Often the first attempt at the architecture of a system in not optimal. During implementation the team learns a lot about the domain. In a second attempt, it will be much more capable of designing an appropriate architecture. Most projects suffering from bad architecture had a good architecture at the outset based on the state of knowledge at that time. However, as the project progressed, it became clear that requirements were misunderstood, and new requirements arose to the point where the initial architecture stopped fitting. Problems arise when this does not lead to changes. If the project just continues with a more and more inappropriate architecture, at some point the architecture will not fit at all. This can be avoided by adjusting the architecture step by step, adapting to the changed requirements based on the current state of knowledge. The ability to change and adjust architecture in line with new requirements is central to this. However, the ability to change the architecture at the level of the entire system is a weakness of microservices while changes within microservices are very simple.
Summary
When using microservices, architecture is even more important than in other systems as it also influences the organization and the ability to independently work on requirements. At the same time, microservices offer many benefits in situations where requirements are unclear and architecture therefore has to be changeable. Unfortunately, the interplay between microservices is hard to modify since the distribution into microservices is quite rigid because of the distributed communication between them. Besides, as microservices can be implemented with different technologies, it gets difficult to move functionality around. On the other hand, changes to individual microservices or their replacement are very simple.
5.3 Infrastructure and Operations
Microservices are supposed to be brought into production independently of each other and should be able to use their own technology stacks. For these reasons each microservice usually resides on its own server. This is the only way to ensure complete technological independence. It is not possible to handle the number of systems required for this approach using hardware servers. Even with virtualization the management of such an environment remains difficult. The number of virtual machines required can be higher than might otherwise be used by the entire IT function of a business. When there are hundreds of microservices, there are also hundreds of virtual machines required, and for some of them, load balancing to distributed work across multiple instances. This requires automation and appropriate infrastructure that is capable of generating a large number of virtual machines.
Continuous Delivery Pipelines
Beyond what is required in production each microservice requires additional infrastructure; it needs its own continuous delivery pipeline so that it can be brought into production independently of other microservices. This means that appropriate test environments and automation scripts are necessary. The large number of pipelines brings about additional challenges: The pipelines have to be built up and maintained. To reduce expense, they also need to be largely standardized.
Monitoring
Each microservice also needs to be monitored. This is the only way to diagnose problems with the service at runtime. With a deployment monolith it is relatively straightforward to monitor the system. When problems arise, the administrator can log into the system and use specific tools to analyze errors. Microservice-based systems contain so many systems that this approach is no longer feasible. Consequently, there has to be a monitoring system that brings monitoring information from all the services together. This information should include not only the typical information from the operating system and the I/O to the hard disc and to the network, but also a view into the application should be possible based on application metrics. This is the only way for developers to find out where the application has to be optimized and where problems exist currently.
Version Control
Finally, every microservice has to be stored under version control independent of other microservices. Only software that is separately versioned can be brought into production individually. When two software modules are versioned together, they should always be brought into production together. If they are not, then a change might have affected both modules—meaning that both services should be newly delivered. Moreover, if an old version of one of the services is in production, it is not clear whether an update is necessary or whether the new version does not contain changes; after all, the new version might only have contained changes in the other microservice.
For deployment monoliths a lower number of servers, environments, and projects in version control would be necessary. This reduces complexity. Operation and infrastructure requirements are much higher in a microservices environment. Dealing with this complexity is the biggest challenge when introducing microservices.
5.4 Conclusion
This chapter discussed the different challenges associated with microservice-based architectures. At a technical level (section 5.1) the challenges mostly revolve around the fact that microservices are distributed systems, which makes ensuring good system performance and reliability more difficult. Technical complexity also increases because of the variety of technologies used. Furthermore, code dependencies can render the independent deployment of microservices impossible.
The architecture of a microservice-based system (section 5.2) is extremely important because of its impact on the organization and the ability to have parallel implementation of multiple stories. At the same time, changes to the interplay of microservices is difficult. Functionality cannot easily be transferred from one microservice to another. Classes within a project can often be moved with support from development tools, but in the case of microservices manual work is necessary. The interface to the code changes—from local calls to communication between microservices—and this increases the effort required. Finally, microservices can be written in different programming languages—in such situations moving code means that it has to be rewritten.
Changes to system architecture are often necessary because of unclear requirements. Even with clear requirements, the team continuously improves its knowledge about the system and its domain. In circumstances where the use of microservices is particularly beneficial because of rapid and independent deployments, architecture should be made especially easy to change. Within microservices changes are indeed easy to implement; however, between microservices they are very laborious.
Finally, infrastructure complexity increases because of the larger number of services (section 5.3) since more servers, more projects in version control, and more continuous delivery pipelines are required. This is a primary challenge encountered with microservice-based architectures.
Part III of the book will show solutions to these challenges.
Essential Points
• Microservices are distributed systems. This makes them technically complex.
• A good architecture is very important because of its impact on the organization. While the architecture is easy to modify within microservices, the interplay between microservices is hard to change.
• Due to the number of microservices, more infrastructure is required—for example, in terms of server environments, continuous delivery pipelines, and projects in version control.