In this chapter, we focused on high-performance system design in a more language-agnostic context. We introduced distributed architectures and explained how they can help scale a platform. We presented some of the challenges that such a paradigm involves, and we focused on solving the problem of shared state inside a cluster. We used CRDTs to implement efficient and performant synchronization among the nodes of a cluster. Using these data types, we were able to simplify our architecture and avoid creating a bottleneck by eliminating the need for a standalone service that is dedicated to storing the shared state. We also kept the latency low by avoiding remote calls on the critical path.

In the second part of this chapter, we analyzed how queues impact latency, and how we can apply load control policies to control latency. By benchmarking the trading strategy pipeline, we discovered the importance of applying back pressure and bounding queue sizes in order to reason about maximum latency. Unbounded queues will eventually lead to disastrous production performance. The formal name for the study of queues is a branch of mathematics known as queueing theory. Queueing theory, like monads, is a topic that deserves a more formal treatment. We focused on using empirical observations to drive improvements. Studying queueing theory will provide you with a stronger theoretical background and the ability to build models for system performance.

We extended the policy of rejecting work to interrupting work that is taking too long. In doing so, we explored a new functional programming technique in the form of the free monad. The free monad allowed us to maintain clean business logic describing what the pipeline does without focusing on how the pipeline accomplishes its goals. This separation of concerns enabled us to also add concurrency to the pipeline without complicating the pipeline description. The principles that we discussed enable you to write high-throughput and low-latency systems that remain robust when the system is at capacity, while retaining an emphasis on functional design.