As discussed in Chapter 6, Messaging, message exchange between services is a key building block for resilient, high-throughput architectures. Brokers are components that host the messages and perform the routing between producers and consumers. One example is Kafka message brokers.

When deploying brokers, it is essential to have a good estimate of the messages-per-second needed for each Topic. This enables us to compute configurations such as the CPU/memory requirements of the brokers and the storage needed for hosting messages.

Another common configuration is the replication factor, which defines the number of nodes on which the message is replicated on. This, in turn, defines how many node failures the Topic can survive and still be able to serve messages to consumers. However, to achieve such reliability, the nodes need to straddle different fault domains. Here, fault domain refers to a group of nodes, which share a common power source and network switch.

Some brokers such as Kafka contain complicated features, such as Group Co-ordinator, which enables highly available message consumption. However, in order to get this right, we need to define tuneables such as heartbeat timeouts (between the brokers and the consumers) correctly.