Chapter 2. Microservice Scenarios
This chapter presents a number of scenarios in which microservices can be useful. Section 2.1 focuses on the modernization of a legacy web application. This is the most common use case for microservices. A very different scenario is discussed in section 2.2. A signaling system is being developed as a distributed system based on microservices. Section 2.3 draws some conclusions and invites the reader to judge the usefulness of microservices in the scenarios presented for themselves.
2.1 Modernizing an E-Commerce Legacy Application
Migrating from a legacy deployment monolith is the most common scenario for microservices. This section starts with a general description of such a scenario and then gets into the details of the legacy application and how to modularize it into microservices.
Scenario
Big Money Online Commerce Inc. runs an e-commerce shop, which is the main source of the company’s revenue. It’s a web application that offers many different functions, such as user registration and administration, product search, an overview of orders, and, of course, the ordering process, the central feature of any e-commerce application.
The application is a deployment monolith: it can only be deployed in its entirety. Whenever a feature is changed, the entire application needs to be deployed anew. The e-commerce shop works together with other systems—for example, with accounting and logistics.
Reasons to Use Microservices
The deployment monolith started out as a well-structured application. However, over the years, more and more dependencies between the individual modules have crept in. This has led to the application becoming very difficult to maintain and update. In addition, the original architecture is no longer suited to the current requirements of the business. Product search, for instance, has been greatly modified as the Big Money Online Commerce Inc. attempts to outperform its competitors in this area. Also, clients have been given a number of self-service options that have helped the company to reduce costs. However, these two modules have become very large, with complex internal structures, and they have numerous dependencies on other modules that had not originally been intended.
Slow Continuous Delivery Pipeline
Big Money has decided to use continuous delivery and has established a continuous delivery pipeline. This pipeline is complicated and slow, as the entire deployment monolith needs to be tested and brought into production in one go. Some of the tests run for hours. A faster pipeline would be highly desirable.
Parallel Work Is Complicated
There are teams working on different new features. However, the parallel work is complicated: the software structure just doesn’t really support it. The individual modules are not separated well enough and have too many interdependencies. As everything can only be deployed together, the entire deployment monolith has to be tested. The deployment and testing phases are a bottleneck. Whenever a team has a problem in the deployment pipeline, all other teams have to wait until the problem has been fixed and the change has been successfully deployed. Also, access to the continuous delivery pipeline has to be coordinated. Only one team can be doing testing and deployment at a time. There has to be coordination between the teams to determine the order in which teams will bring their changes into production.
Bottleneck During Testing
In addition to deployment, tests also have to be coordinated. When the deployment monolith runs an integration test, only the changes made by one team are allowed to be contained in the test. There were attempts to test several changes at once. This meant it was very hard to discern the origin of errors and led to error analyses that were long and complex.
One integration test requires approximately one hour. About six integration tests are feasible per working day, because errors have to be fixed and the environment has to be set up again for the next test. If there are ten teams, one team can bring one change into production every two days, on average. However, often a team also has to do error analysis, which lengthens integration. For that reason, some teams use feature branches in order to separate themselves from integration; they perform their changes on a separate branch in the version control system. Integrating these changes into the main branch later on often causes problems; changes are erroneously removed upon merging, or the software suddenly contains errors that are caused by the separated development process and that only show up after integration. These errors can only be eliminated in lengthy processes after integration.
Consequently, the teams slow each other down due to the testing (see Figure 2.1). Although each team develops its own modules, they all work on the same code base so that they impede each other. As a consequence of the shared continuous delivery pipeline and the ensuing need for coordination, the teams are unable to work either independently of each other or in parallel.
Approach
Because of the many problems being experienced, Big Money Online Commerce Inc. decided to split off small microservices from the deployment monolith. The microservices each implement one feature, such as the product search, and are developed by individual teams. Each team has complete responsibility for an individual microservice, starting from requirements engineering up to running the application in production. The microservices communicate with the monolith and other microservices via REST. The client GUI is also divided between the individual microservices based on use cases. Each microservice delivers the HTML pages for its use cases. Links are allowed between the HTML pages of the microservices. However, access to the database tables of other microservices or the deployment monolith is not allowed. Integration of services is exclusively done via REST or via links between the HTML pages.
The microservices can be deployed independently of each other. This enables changes in a microservice to be deployed without the need to coordinate with other microservices or teams. This greatly facilitates parallel work on features while reducing coordination efforts.
The deployment monolith is subject to far fewer changes due to the addition of microservices. For many features, changes to the monolith are no longer necessary. Thus, the deployment monolith is changed and deployed less often. Originally, the plan was to completely replace the deployment monolith at some point. However, in the meantime it seems more likely that the deployment monolith will just be deployed less and less frequently because most changes take place within the microservices. Thus the deployment monolith does not disturb work anymore. To replace it entirely is not necessary and also does not appear sensible in economic terms.
Challenges
Implementing microservices creates additional complexity at the start; all the microservices need their own infrastructure, and at the same time, the Monolith has to be supported.
The microservices require a lot more servers and therefore pose very different challenges. Monitoring and log file processing has to deal with the fact that data originates from different servers. As a result, information has to be centrally consolidated. A substantially larger number of servers must be handled, not only in production, but also in the different test stages and team environments. This is only possible with good infrastructure automation. It is necessary to support different types of infrastructure for the monolith and the microservices, and this leads to substantially more servers overall.
Entire Migration Lengthy
The added complexity due to the two different software types will persist for a long time, as it is a very lengthy process to completely migrate away from the monolith. If the monolith is never entirely replaced, the additional infrastructure costs will remain as well.
Testing Remains a Challenge
Testing is an additional challenge; previously, the entire deployment monolith was tested in the deployment pipeline. These tests are complex and take a long time, as all the functions of the deployment monolith have to be tested. If every change to every microservice is sent through these tests, it will take a long time for each change to reach production. Additionally, the changes have to be coordinated, because each change should be tested in isolation so that errors can be easily linked back to the change that caused them. In that scenario, a microservices-based architecture does not seem to have major advantages over a deployment monolith; while microservices can in principle be deployed independently of each other, the test stages preceding deployment still have to be coordinated, and each change still has to pass through them individually.
Current Status of Migration
Figure 2.2 presents the current status; product search works as an independent microservice and is completely independent of the deployment monolith. Coordination with other teams is hardly ever necessary. Only in the last stage of the deployment do the deployment monolith and the microservices have to be tested together. Each change to the monolith or any microservice has to run through this step. This causes a bottleneck. The team “Customer” works together with the team “Order Process” on the deployment monolith. In spite of microservices, these teams still have to closely coordinate their work. For that reason, the team “Order Process” has implemented its own microservice, which forms part of the order process. In this part of the system, changes can be introduced faster than in the deployment monolith, not only due to the younger code base, but also because it is no longer necessary to coordinate with the other teams.
Creating Teams
For the teams to be able to work independently on features, it is important to create teams that match to functionalities such as product search, customer processing, or order processing. If teams correspond to technical layers such as the UI, middle tier, or database instead, each feature requires the involvement of all the teams, because a feature normally comprises changes to the UI, middle tier, and database. Thus, to minimize coordination efforts between the teams, the best approach is to create teams that center around features like product search. Microservices support the independence of the teams by their own technical independence from each other. Consequently, teams need to coordinate less in respect to basic technologies and technical designs.
The tests also need to be modularized. Each test should ideally deal with a single microservice so that it is sufficient to perform the test when changes are made in the respective microservice. In addition, it might be possible to implement the test as unit test rather than as an integration test. This progressively shortens the test phase in which all microservices and the monolith have to be tested together. This reduces the coordination problems for the final test phase.
Migrating to a microservices-based architecture created a number of performance problems and also some problems due to network failures. However, these problems were solved over time.
Advantages
Thanks to the new architecture, changes can be deployed much faster. A team can bring a change into production within 30 minutes. The deployment monolith, on the other hand, is deployed only weekly because the tests are not yet fully automated.
Deploying the microservices is not only much faster, but also much less risky: less coordination is required. Errors are more easily found and fixed because developers still know what they have been working on well, as it was only 30 minutes ago.
In summary, the goal was attained; the developers can introduce more changes to the e-commerce shop. This is possible because the teams need to coordinate their work less and because the deployment of a microservice can take place independently of the other services.
The option of using different technologies was only sparingly used by the teams. The previously used technology stack proved sufficient, and the teams wanted to avoid the additional complexity caused by the use of different technologies. However, the long-needed search engine for the product search was introduced. The team responsible for product search was able to implement this change on its own. Previously, the introduction of this new technology had been prohibited because the associated risk had been considered too great. In addition, some teams have new versions of the libraries of the technology stack in production because they needed the bug fixes included in the more recent version. This did not require any coordination with the other teams.
Conclusion
Replacing a monolith via the implementation of microservices is a very common scenario for the introduction of microservices. It requires a lot of effort to keep developing a monolith and to add new features to it. The complexity of the monolith and the associated problems caused by it progressively increase over time. It is often very difficult and risky to completely replace an existing system with a newly written one.
Rapid and Independent Development of New Features
In the case of companies like Big Money Online Commerce Inc., the rapid development of new features and the ability to do parallel work on several features are vital for the success of the business. Only by providing state-of-the-art features can new customers be won and existing customers be kept from switching to other companies. The promise of being able to develop more features faster makes microservices compelling in many use cases.
Influence on the Organization
The presented example illustrates the influence of microservices on the organization. The teams work on their own microservices. As the microservices can be developed and deployed independently of each other, the work of the different teams is no longer linked. In order to keep it that way, a microservice should not be changed by more than one team at any time. The microservices architecture requires a team organization corresponding to the different microservices. Each team is responsible for one or several microservices, each of which implements an isolated piece of functionality. This relationship between organization and architecture is especially important in the case of microservices-based architectures. Each team takes care of all issues concerning “its” microservices from requirements engineering up to operation monitoring. Of course, for operation, the teams can use common infrastructure services for logging and monitoring.
And finally, if the goal is to achieve a simple and fast deployment in production, just including microservices in the architecture will not be sufficient. The entire continuous delivery pipeline has to be checked for potential obstacles, and these have to be removed. This is illustrated by the tests in the presented example; the testing of all microservices together should be reduced to the essential minimum. Each change has to run through an integration test with the other microservices, but this test must run quickly to avoid a bottleneck in integration tests.
Amazon Has Been Doing It for a Long Time
The example scenario presented here is very similar to what Amazon has been doing for a very long time, and for the discussed reasons: Amazon wants to be able to quickly and easily implement new features on its website. In 2006, Amazon not only presented its Cloud platform, but also discussed how it develops software. Essential features are:
• The application is divided into different services.
• Each service provides a part of the website. For instance, there is a service for searching, and another one for recommendations. In the end, the individual services are presented together in the UI.
• There is always one team responsible for one service. The team takes care of developing new features as well as operating the service. The idea is: “You build it—you run it!”
• The Cloud platform (i.e., virtual machines) acts as the common foundation of all services. Apart from that, there are no other standards. As a result of this, each team is very free in their choice of technologies.
By introducing this type of architecture, Amazon implemented the fundamental characteristics of microservices back in 2006. Moreover, Amazon introduced DevOps by having teams consisting of operation experts and developers. This approach means that deployments occur largely in an automated fashion, as the manual construction of servers is not feasible in Cloud environments. Therefore, Amazon also implemented at least one aspect of continuous delivery.
In conclusion, some companies have been using microservices for a number of years already—especially companies with an Internet-based business model. This approach has already proven its practical advantages in real life. In addition, microservices work well with other modern software practices such as continuous delivery, Cloud, and DevOps.
2.2 Developing a New Signaling System
Greenfield applications can also be built using microservices. In some cases, that is the much more natural approach. This section starts with a general description of a greenfield scenario and then gets into the details of the example—a new signaling system.
Scenario
Searching for airplanes and ships that have gone missing is a complex task. Rapid action can save lives. Therefore, different systems are required. Some provide signals such as radio or radar signals. These signals have to be recorded and processed. Radio signals, for example, can be used to obtain a bearing, which subsequently has to be checked against radar-based pictures. Finally, humans have to further analyze the information. The data analyses, as well as the raw data, have to be provided to the different rescue teams. Figure 2.3 provides an overview of the signaling system. Signal Inc. builds systems for exactly these use cases. The systems are individually assembled, configured, and adapted to the specific needs of the respective client.
Reasons to Use Microservices
The system is composed of different components that run on different computers. The sensors are distributed all over the area to be monitored and are provided with their own servers. However, these computers are not supposed to handle the more detailed data processing or store the data. Their hardware is not sufficiently powerful for that. Data privacy considerations would also render such an approach very undesirable.
Distributed System
For these reasons, the system has to be a distributed system. The different pieces of functionality are distributed within the network. The system is potentially unreliable, as individual components and the communication between components can fail.
It would be possible to implement a large part of the system within a deployment monolith. However, upon closer consideration, the different parts of the system have to fulfil very different demands. Data processing requires a substantial CPU and an approach that enables numerous algorithms to process the data. For such purposes, there are solutions that read events out of a data or event stream and process them. Data storage requires a very different focus. Basically, the data has to be maintained within a data structure that is suitable for different data analyses. Modern NoSQL databases are well suited for this. Recent data is more important than old data. It has to be accessible faster, while old data can even be deleted at some point. For final analysis by experts, the data has to be read from the database and processed.
Technology Stack per Team
Each of the discussed tasks poses different challenges. Consequently, each requires not only a well-adapted technology stack, but also a dedicated team consisting of technical experts on the respective task. Additionally, people are needed to decide which features Signal Inc. will bring to the market and to define new requirements for the systems. Systems for processing and sensors are individual products that can be positioned on the market independently of each other.
Integration of Other Systems
An additional reason for the use of microservices is the opportunity to easily integrate other systems. Sensors and computing units are also provided by other companies. The ability to integrate such solutions is a frequent requirement in client projects. Microservices enable the easy integration of other systems, as the integration of different distributed components is already a core feature of a microservices-based architecture.
For these reasons, the architects of Signal Inc. decided to implement a distributed system. Each team must implement its respective domain in several small microservices. This approach should ensure that microservices can be easily exchanged, and the integration of other systems will be straightforward.
Only the communication infrastructure to be used by all services for their communication is predetermined. The communication technology supports many programming languages and platforms so that there are no limitations as to which concrete technology is used. To make flawless communication possible, the interfaces between the microservices have to be clearly defined.
Challenges
A failure of communication between the different microservices presents an important challenge. The system has to stay usable even if network failures occur. This requires the use of technologies that can handle such failures. However, technologies alone will not solve this problem. It has to be decided as part of the user requirements what should happen if a system fails. If, for instance, old data is sufficient, caches can be helpful. In addition, it may be possible to use a simpler algorithm that does not require calls to other systems.
High Technological Complexity
The technological complexity of the entire system is very high. Different technologies are employed to satisfy the demands of the different components. The teams working on the individual systems can make largely independent technology decisions. This enables them to always implement the most suitable solution.
Unfortunately, this also means that developers can no longer easily switch between teams. For example, when there was a lot of work for the data storage team, developers from other teams could hardly help out, as they were not even proficient in the programming languages the data storage team was using and did not know the specific technologies, such as the used database.
It can be a challenge to run a system made up of so many technologies. For this reason, there is one standardization in this area: all microservices must be able to be run in a largely identical manner. They are virtual machines so that their installation is fairly simple. Furthermore, the monitoring is standardized, which determines data formats and technologies. This makes the central monitoring of the applications possible. In addition to the typical operational monitoring, there is also monitoring of application-specific values, and finally an analysis of log files.
Advantages
In this context, the main advantage offered by microservices is good support for the distributed nature of the system. The sensors are at different locations, so a centralized system is not sensible. The architecture has adapted to this fact by further dividing the system into small microservices that are distributed within the network. This enhances the exchangeability of the microservices. The microservices approach supports the technology diversity, which characterizes this system.
In this scenario, time-to-market is not as important as in the e-commerce scenario. It would also be hard to implement, as the systems are installed for different clients and cannot be easily reinstalled. However, some ideas from the continuous delivery field are used: for instance, the largely uniform installation and the central monitoring.
Verdict
Microservices are a suitable architectural pattern for this scenario. The system can benefit from the fact that typical problems can be solved during implementation by established approaches from the microservices field: for example, technology complexity and platform operation.
Still, this scenario wouldn’t be immediately associated with the term “microservice.” This leads to the following conclusions:
• Microservices have a wider application than is apparent at first glance. Outside of web-based business models, microservices can solve many problems, even if those issues are very different from the ones found in web companies.
• Indeed, many projects from different fields have been using microservice-based approaches for some time, even if they do not call them by this name or only implement them partially.
• With the help of microservices, these projects can use technologies that are currently being created in the microservice field. In addition, they can benefit from the experiences of others who have worked in this field, for instance in regards to architecture.
2.3 Conclusion
This chapter presented two very different scenarios from two completely distinct business areas: a web system with a strong focus on rapid time-to-market, and a system for signal processing that is inherently distributed. The architectural principles are very similar for the two systems, although they originate from different reasons.
In addition, there are a number of common approaches, among those the creation of teams according to microservices and the demands in regards to infrastructure automatization, as well as other organizational topics. However, in other areas, there are also differences. For the signaling system, it is essential to have the option to use different technologies, as this system has to employ a number of different technologies. For the web system, this aspect is not as important. Here, the independent development, the fast and easy deployment, and finally the better time-to-market are the critical factors.
Essential Points
• Microservices offer a significant number of advantages.
• In the case of web-based applications, continuous delivery and short time-to-market can be important motivations for the use of microservices.
• However, there are also very different use cases for which microservices as distributed systems are extremely well suited.