Modern systems demand a lot from the storage systems. There is a need to scale storage systems in terms of queries per second (QPS), the number of concurrent connections, the size of the data, and so on. Also, many applications need ultra-fast requests for a few use cases. While relational systems have, and continue to provide, a reliable persistence technology, the traditional scale-up approach, that is, using better hardware equipment, is not able to keep up with requirements. It is extremely difficult to provide ACID semantics in distributed systems, making scale-out for relational databases a difficult task. Note that there are mechanisms such as distributed transactions, but using them is very complex and they generally lead to very fragile systems. Joins are particularly inefficient in distributed databases. In single-instance databases, joins are efficiently handled using indices and data locality. In distributed nodes, joins will need movement of data across the network to execute the necessary comparison operations. These inefficiencies cause distributed joins to be a inefficient compared to single-node systems.

We will do a deep-dive on scaling data later in this chapter, but in this section we wil introduce a new class of storage systems, called key-value stores.