Microservices Versus Monoliths

Today, there are two often-discussed approaches to building applications. Applications can either be monolithic or they can be built using microservices. Both approaches exist on either end of a spectrum on which most software projects fall somewhere in between. Very few applications are completely monolithic or perfect microservices.

Monolithic applications are built such that all of the components of the application are deployed as a single unit. Complexity in monolithic applications is isolated by creating libraries that are then knitted together into the larger whole.

Microservices, on the other hand, are built by dividing the larger application into smaller, specialized services. These microservices perform very small tasks in an isolated fashion. Complexity is isolated to the individual microservices and then put together using other microservices.

Availability is not really a question of monoliths or microservices. You can create highly available applications using either approach. However, the manner in which you do that, and the effects on the scalability of your application, are dependent on which route you choose.

In a monolithic application there is only a single deployable unit. In this case, the way we provide availability—and scalability—is to simply deploy more units. If any one unit is lost, others will pick up the slack. Akka doesn’t really help us here. The deployed units don’t need to communicate with one another, so there is no need to introduce features like Akka Cluster. Within the monolith, we can use Akka to help us make better use of resources. We can also use it to help isolate faults by using bulkheading and other techniques. Akka does have a place within a monolithic system, but when we are speaking of availability using Akka, we are more interested in techniques that don’t fit well in a monolithic application.

Providing availability with microservices is similar to monoliths in that the way to provide it is by deploying multiple instances of each microservice. However, in this case, because each microservice is separate, it can be deployed independently, scaled independently, and our availability needs can be tuned independently. Some areas of a system might be critical and need to be highly available. Other areas might be less important, perhaps only needing to run for a few hours a day or even less. In these cases, availability might be much less of a concern.

When we have multiple microservices that need to communicate with one another, Akka provides facilities that can help. But, before we go into detail about what facilities Akka provides, let’s first take a moment to talk about how we divide up an application into microservices.

Bounded Contexts as Microservices

Domain-driven design (DDD) gives us a very natural way to divide an application into smaller pieces. Bounded contexts are excellent boundaries on which to divide. By definition, bounded contexts isolate areas of the domain from one another. Therefore, following this idea, we can isolate areas of our application along the same dividing lines. This means separating those areas out into separate microservices.

Sometimes, those microservices might have some shared elements, and you might need some common libraries in certain cases to share code. In DDD, bounded contexts can share domain elements through what is known as a shared kernel. This shared kernel is basically a set of common domain elements that can be used by multiple bounded contexts. In Akka, the message protocol that is passed between clustered applications is commonly found in the shared kernel.

This message protocol, as defined in the shared kernel, represents the API for the bounded context or microservice. It defines the inputs and outputs for that microservice. Depending on the mechanism of communication being used between the microservices, it might consist of a series of simple case classes that are directly sent to other actors, or it might represent case classes that are first serialized to JSON or some other format and then sent via an HTTP call to an Akka HTTP endpoint.

In our scheduling domain example, we have three bounded contexts defined: the User Management context, the Scheduling context, and the Skills context. We can separate each of these into microservices. We can then define either an HTTP endpoint to interact with those services, or we can directly send messages to the actors using Akka Cluster or Akka Remoting.

After we have successfully partitioned our application into microservices, we can then begin thinking about how we want to scale and deploy the application. Certain microservices might need many copies in order to provide scalability, whereas others might, by necessity, be limited to a single copy. Depending on the criteria, this can have an effect on how we make our application available.

Fine-Grained Microservices

Although bounded contexts are a good starting point for dividing up an application, sometimes that’s not enough. Sometimes, we need to break things into chunks smaller than a bounded context. A common pattern in Akka is to separate the interface and the domain into separate microservices. In the case of a REST API, that might mean that the REST API exists as a single microservice, whereas the actual logic behind that API could be another. This is advantageous because it makes it possible to scale the two sides independently.

The benefit of Akka is that you can make those types of decisions without affecting the architecture of the application. Location transparency means that we can decide whether we want our REST API in the same deployable unit, or in a separate deployable unit without affecting the basic structure of the application.

In the following section, we will explore several techniques that you can use to break apart an application, all toward the goal of providing the availability and scalability that you want. Which technique is best suited for you depends on the specifics of your system’s design.

Cluster-Aware Routers

When building using a single, local actor system, routers are a scalability feature. With them, you can process more messages in parallel which makes it possible to scale up an application within a single machine. However, when dealing with a distributed system, you can also use routers as an availability feature. Cluster-aware routers are like normal routers, except that their routees might reside on other nodes in the cluster. This allows those routers to provide availability on top of scalability.

With a cluster-aware router, you can spin up multiple nodes in your cluster. These nodes can act as hosts to the routees of the router. Then, within your application, when you want to send a message to the actors, you can send it through the cluster-aware router. The router will then ensure that one of the nodes will receive the message, depending on which routing strategy you adopted.

If a node fails, the router can simply direct the message to a different, active node. From the client perspective, you don’t need to worry about which node receives the message; the router takes care of that detail for you. But you now have the benefit of added availability, given that nodes can leave the cluster without the client even realizing it. In fact, if you are using a pool router, when a node does leave the cluster, the pool router can even create new routees on other nodes in the cluster in order to compensate for the loss of that node.

One thing to be aware of is that the routing strategies work differently when using cluster-aware routers than they do with local routers. For example, with a smallest-mailbox router, no information is passed between nodes regarding the mailbox size. This means that the smallest mailbox router has no idea which node in the cluster has a smaller mailbox. In this case, the router uses the normal smallest-mailbox logic for all local routees, but remote routees are given lower priority.

So where might you use a cluster-aware router? Cluster-aware routers are a much more general-purpose construct than a cluster singleton, so their use cases are more varied. But a good example of where you might use this is in a worker pattern. In this case, you have a single actor that will push tasks to multiple workers. In our sample domain, for example, we might use the worker pattern to make schedule adjustments. We want to be able to do several schedule adjustments in parallel, particularly because the amount of work involved might be significant. In this case, we can push a message into a router, which will then distribute it to an available worker. Those workers can reside on any node in the cluster. If a node is lost, the workers are lost as well, but depending on the router configuration, they might be reallocated to another node in the cluster.

In practice, this worker pattern might look something like the following:

object Scheduler {
  case class ScheduleProject(project: Project)
  case class ProjectScheduled(project: Project)

  def props() = Props(new Scheduler)
}

class Scheduler extends Actor {
  import Scheduler._

  override def receive: Receive = {
    case ScheduleProject(project) =>
      // Do Work
      sender() ! ProjectScheduled(project)
  }
}

Again, as with the cluster singleton, there is no evidence here that we are going to use Akka Cluster. We can decide that later.

Indeed, creating instances of the workers is no different than creating instances of local actors:

val scheduler = system.actorOf(Scheduler.props(), "scheduler")

This example creates an instance of the actor and names it scheduler. Within your local system, you can treat this just as you would any other actor, because, in fact, it is just a standard actor. The magic happens on another clustered node when you want to refer to this actor:

val scheduler = system.actorOf(
    ClusterRouterGroup(RoundRobinGroup(Nil), ClusterRouterGroupSettings(
      totalInstances = 100,
      routeesPaths = immutable.Seq("/user/scheduler"),
      allowLocalRoutees = false,
      useRole = None
    )).props(),
    "scheduler"
  )

This code configures a cluster-aware group router that references the scheduler. You can see that it has a path to the scheduler, which matches the path that we created for the instance of the actor to which we want to route. In this case, we are allowing up to 100 instances to be used by this group router, but we have only created a single instance. If we had 100 nodes, though, each node could be creating and managing an instance of the scheduler. We could also specify a different sequence of paths for the routee’s paths in case the pattern changes from node to node.

This router will now do a lookup on the cluster nodes to find any instances of the scheduler. It will then route messages to those instances—in this case, using a round-robin routing strategy. One thing to note, however, is that unlike the singleton proxy, the router does not buffer messages until a node can be found. This means that if you try to send a message before your node has connected to the cluster and found the routees, that message will be lost. You can help mitigate this by listening to cluster events and only starting to send messages after your cluster is fully established. Of course, Akka’s delivery guarantee is still only At Most Once, so if you really need reliable delivery, you will need to use one of the other techniques to achieve that.

Distributed Data

Within an eventually consistent, distributed system, such as the one we are building, we sometimes encounter situations in which there is transient data. This data is not something that needs to be saved in a database. It exists only for the life of the application. If the application is terminated, it is safe to terminate that data, as well. This could include things like user session information. When a user logs in, you might need to store information about that user. When did she log in? What security token did she use? What was the last activity she performed? Information like that is interesting, but keeping it isn’t always valuable. After the user has logged out, that information becomes irrelevant and potentially needs to be cleaned up.

At the same time, it might be important that the transient information be available on all nodes. If a user is experiencing network problems and she becomes disconnected and then reconnected, she might end up reconnected to a different node. If that information is unavailable, the system loses its value. What this situation calls for is a way to maintain this data across several nodes in a cluster without saving it to a database.

In fact, there is a way. If you can represent that data by using data structures that follow specific criteria, you can replicate it reliably in an eventually consistent fashion. This replication can happen entirely in memory; no database needs to be involved. This gives you a distributed, eventually consistent method of storing and retrieving data.

These eventually consistent data types are called Conflict-Free Replicated Data Types, or CRDTs. CRDTs are an emerging concept. The idea first appeared in a paper in 2011 by Marc Shapiro et al. At this point, CRDTs are not widely used, but interest in them is growing, particularly for use cases in which a system handles very high volumes of data traffic but still needs to be performant.

Akka has its own implementation of CRDTs called Distributed Data. This is a new module in Akka, and it is still experiencing some flux. The API is changing as developers iron out the details of how it should work.

The basic idea behind CRDTs in Akka is that you have certain data types. These include Counters, Sets, Maps, and Registers. To be considered CRDTs, these data types must include a conflict-free merge function. This merge function has the job of taking two different states of the data (coming from two different locations in the cluster) and merging them together to create a final result. If this merge can be done without conflict, you can use this data structure to replicate across nodes.

Here’s how it works. As each node receives updates to its data, it broadcasts its current state to other nodes. Other nodes receive the updated state, merge it with their own state, and then store the end result.

CRDTs typically work by storing extra information along with the state. Additive operations are often safe, but removal operations become more complex. For example, what happens if you try to remove an element from a Set that has not yet received the update to add that element? How do you resolve that conflict? This is typically handled by making removal an additive operation. Rather than removing an item, you mark that item as removed, but it is still present in the data. This means that your data continually grows. Even as you try to remove information, you are in fact adding information. There are optimizations that you can apply in certain circumstances to help with this.

So how do you use Akka Distributed Data? Let’s take a look at how you might replicate some simple session information among nodes in the cluster. In this example, let’s assume that you are replicating only the session IDs, which are a custom type that wraps a UUID. We use the ORSet data type for replication. An ORSet, or Observed Remove Set, is a special type of Set that uses a version vector to keep track of the creation of elements. This version vector is then used as part of the merge function to determine causality. In an ORSet, if an add and a remove happen out of order, or concurrently, the version vector will be used to resolve the conflict. You can’t remove a record if you haven’t yet seen the add for it. If an add and remove happen concurrently, the add wins. Here’s how it looks:

case class SessionId(value: UUID = UUID.randomUUID())

object SessionManager {
  case class CreateSession(sessionId: SessionId)
  case class SessionCreated(sessionId: SessionId)

  case class TerminateSession(sessionId: SessionId)
  case class SessionTerminated(sessionid: SessionId)

  case object GetSessionIds
  case class SessionIds(ids: Set[SessionId])

  def props() = Props(new SessionManager)
}

class SessionManager extends Actor {
  import SessionManager._

  private val replicator = DistributedData(context.system).replicator
  private val sessionIdsKey = ORSetKey[SessionId]("SessionIds")
  private implicit val cluster = Cluster(context.system)

  override def receive: Receive = {
  case CreateSession(id) =>
    replicator ! Update(
      sessionIdsKey,
      ORSet.empty[SessionId],
      WriteLocal,
      request = Some(sender() -> SessionCreated(id))
    ) {
      existingIds =>
        existingIds + id
    }

  case UpdateSuccess(
    `sessionIdsKey`,
    Some((originalSender: ActorRef, response: SessionCreated))
  ) => originalSender ! response

  case TerminateSession(id) =>
    replicator ! Update(
      sessionIdsKey,
      ORSet.empty[SessionId],
      WriteLocal,
      request = Some(sender() -> SessionTerminated(id))
    ) {
      existingIds =>
        existingIds - id
    }

  case UpdateSuccess(
    `sessionIdsKey`,
    Some((originalSender: ActorRef, response: SessionTerminated))
  ) => originalSender ! response

  case GetSessionIds =>
    replicator ! Get(sessionIdsKey, ReadLocal, request = Some(sender()))

  case result @ GetSuccess(`sessionIdsKey`, Some(originalSender: ActorRef)) =>
    originalSender ! SessionIds(result.get(sessionIdsKey).elements)
  }

}

You can see from this code that in order to use data replication, you need to access the replicator. This special actor can be obtained by using the DataReplication extension. You can update replicated data by sending an Update message to the replicator. As soon as the data is updated, you will receive an UpdateSuccess message. You can retrieve replicated data by sending a Get message to the replicator. The replicator will respond with a GetSuccess message.

There are other messages that you can send to the replicator, as well. You can subscribe to updates so that you will be notified when a value changes by using the Subscribe message, and you can delete records by using the Delete message.

The replicator will take care of replicating your state across the cluster. This actor will be eventually consistent across the cluster. This means on any node in the cluster that runs the SessionManager, you can request the list of SessionIds, and you will get a list of active sessions. Of course, because this is eventually consistent, the nodes might give you slightly different lists. You can control this eventual consistency by specifying different read/write consistency values.

In the previous example, local consistency is specified, which means the replicator only looks at the local node. However, you can specify more strict values that will cause the replicator to ensure that a certain number of nodes must agree on the result before you consider it valid. ReadAll and WriteAll means that all nodes must agree, but it also affects your availability. If a node is down, the replicator won’t be able to reach the required consistency and your request will fail. You can also specify Read​Ma⁠jority or WriteMajority, which will ensure that the majority of nodes agree on the value. This gives you a nice balance of consistency versus availability.

Graceful Degradation

One of the benefits of breaking your application into microservices is that it enables graceful degradation. Within an application, we enable graceful degradation by setting up failure zones in the application. By creating actors in hierarchies that allow a section of the application to fail without bringing down the entire system, we allow our application to degrade in pieces rather than failing all at once. Microservices enable this same behavior but spread across multiple Java Virtual Machines (JVMs) and potentially multiple machines.

We would like to avoid the situation implied in the old quote “If at first you don’t succeed, then perhaps skydiving is not for you.” We want our applications to continue even in the face of failure.

In our sample scheduling domain, we can separate out a few services. We have the scheduling engine, the Project Management service, the Person Management service, and the Skills service. This means that if our scheduling engine fails, it doesn’t prevent us from adding or removing projects or people. It only prevents us from scheduling people on a project. You can do this within a single monolithic application, by using actors to create failure zones, or you can do it by using microservices.

Graceful degradation means that while portions of an application might fail, the application as a whole can continue to operate, keeping it available even in the face of failure. It also means that noncritical portions of the application can be taken down, perhaps for maintenance or other reasons, without necessarily affecting your users in an adverse way.

Let’s take a look at that. The scheduling engine in our system doesn’t necessarily need to respond in a rapid fashion. It might be reasonable for a new project to be created but for it to take a period of time, minutes, or maybe even hours, before it returns results. There is no expectation that the moment a project is created it should be scheduled, as well. This means that if that section of the application were in need of some maintenance, perhaps a database upgrade, you could take the entire application down and perform that upgrade. In the meantime, users can still add new people and they can still add new projects; they just won’t get the schedule for that project until after the maintenance is completed. And that’s OK.

You can use the Circuit Breaker pattern discussed in Chapter 7 to provide graceful degradation. Typically, detection of the failure of an external system is a time-consuming operation. The system might need to wait for a connection to time out. If it did this for every subsequent request until the resource became available again, the system as a whole would take on an additional burden. By using the Circuit Breaker, the system is able to detect the first time a problem occurs and then quickly fail any additional requests until the timeout has elapsed. This helps keep subsequent failing requests from taking longer than they need to, and it also reduces the load on the system so that it can recover. This improves availability because rather than presenting timeout alerts, which can be time consuming, the system instead can quickly inform you about the error and which service is unavailable. Even though a portion of the system is unavailable, it is still responsive, even if all it’s doing is notifying you of the error.

Deployment

After you have built your application to support availability using the techniques outlined, you then need to be able to deploy it in a way that maintains that availability. If the deployment process requires that you bring down the entire application suite, you haven’t achieved the desired goal.

With a bit of planning and forethought, though, you can come very close to 100 percent service availability, even during deployment.

In Akka, each executable process typically contains a single actor system, which joins the remainder of the processes in the cluster to create a single distributed actor system.

Staged Deployment/Rolling Restarts

One of the most common reasons for nodes to become unavailable in a system isn’t an error at all. Rather, it’s a normal operational concern: upgrades. When you have a new version of your application or service, you need to deploy it to your production environment at some point.

Automation can be critical here, particularly when you’re dealing with microservices. You will not have just a single copy of a monolithic application to deploy, but rather many copies of numerous microservices, potentially even hundreds in a large system. This would be a tedious and error-prone process if it were a manual operation, but with a bit of automation, you can make it simpler and more convenient.

The most commonly used approach with distributed systems is to use staged deployments, also called rolling restarts. The basic process for a rolling restart looks like this:

1. Select a node in the cluster.

2. Stop routing new traffic to this node.

3. Allow any existing requests to complete. This is sometimes called “draining” the node.

4. Gracefully terminate the application you are planning to upgrade. In an Akka Cluster, this would involve removing the node from the cluster.

5. Copy the new executable to the node.

6. Launch the new executable. In an Akka Cluster, the node would then need to join the cluster.

7. Verify that the node is available and that it is the expected version (a monitoring status page can help here).

8. Begin routing traffic to the new node.

9. Repeat these steps for other nodes.

Rolling restarts require that you have multiple instances of a service running in order to maintain availability. When the node is taken down for deployment, other nodes need to be available to pick up the slack.

If you’re using a cluster singleton, you can’t maintain availability during an upgrade—the singleton pattern doesn’t allow it. However, you can minimize the downtime by allowing other nodes to host the singleton. In this case, when you bring down the current node, the singleton can fail-over to one of the other hosts, exposing only a very short period of downtime.

Blue/Green Deployment

An alternative to the rolling upgrade process is called blue/green deployment. For this method, you must have more nodes then you actually need. Usually you would have double the number of nodes that are required for normal operation, with only half of the nodes actually servicing requests at any given time.

In this deployment model, you designate 50 percent of the nodes as “blue,” and the other 50 percent as “green.” Suppose that your green nodes are currently online and servicing requests. You can then shut down all the blue nodes, upgrade them, bring them back up, and then check them for operational status (remember, no traffic is being routed to them) The final step is to swap the blue set for the green set, making the blue nodes active, and the green nodes inactive. Now you are free to upgrade the green set using the same process.

One big advantage to this process is that if a problem arises with the newly activated set after the switchover, you can simply switch back to the previous set (in this example, the green set) before upgrading them. For this reason it can be beneficial to keep your inactive set on the old version for a period of time while the new version runs, and watch for any issues. Only when you are confident that the new version is stable and working would you upgrade the inactive set.

Crash Recovery/Operational Monitoring

A critical element of availability in a clustered environment is the ability to recognize when a failure has occurred and then respond to it appropriately. Even better is when we can recognize the problem before it occurs and take steps to prevent it. To achieve this, we need to do operational monitoring. There are a variety of monitoring tools that you can use, and each of them has its place in a system.

case class Symptom(description: String)

sealed trait Diagnosis {
  def name: String
}

case class Healthy(name: String) extends Diagnosis

case class Unhealthy(name: String, symptoms: Set[Symptom]) extends Diagnosis

case class HealthReport(versionNumber: String, healthChecks: Seq[Diagnosis])

trait HealthCheck {
  def checkHealth(): Diagnosis
}

class HealthCheckRouting(applicationVersion: String, checks: Seq[HealthCheck])
  extends HealthReportProtocol {
  val routes = {
    path("health") {
      get {
        complete {
          HealthReport(applicationVersion, checks.map(_.checkHealth()))
        }
      }
    }
  }
}

Conclusion

We have now shown how actor-based systems have different requirements: just logging is not enough, and static testing before deployment is insufficient as well.

With proper monitoring, however, availability can be ensured, and the overall health of your system made immediately visible despite rapidly changing load (even in the face of the system itself being continually upgraded).

Having all of these benefits while still retaining high performance, however, requires careful attention. This will be the topic of Chapter 9, as we add the final piece you need for building your own actor-based systems.