The problems just described were noticed at Netflix; the engineering team there developed a set of design patterns (and implementations in Java) called Hystrix to solve these problems.

The key idea is to wrap the dependency calls in command objects from Netflix. In Java, these commands are executed off the main request handling thread and delegated to a dependency-specific thread pool. This allows the Bulk heading of requests, that is, Dependency X down will only block all threads of the thread pool allocated to dependency-x. Other resources will be insulated and other requests that don't involve dependency-x can continue to be handled.

This is depicted in the following diagram:

A dependency that responds slowly or with variable latencies is much worse than a service that fails fast, as the former causes resources to be hogged while waiting. To avoid this, the Hystrix set of patterns includes the Timeout concept. Each dependency is configured with a timeout (typically the 99.5 percentile of expected response latency) so that there is a worst-case time for which resources are blocked for a down service. If the timeout occurs before the service responds, then failure of the dependency is assumed.

The following sections cover Hystrix in detail.