Scaling a service by itself is not sufficient. Just because you create instances of your service doesn't mean they'll just work together. There are two important steps to enable a service scale:

• Making instances work together
• Making instances reachable

Instances don't need to communicate with each other, but an instance must be able to work without interfering with the others. An instance should be able to handle a request from a client that was previously served by another instance. In our case, this means an instance must be able to manipulate an image that was uploaded by another instance. This ensures that if an instance goes offline, others can take over its clients.

Instances need to be reachable, which means you need to be able to point your customers to the nearest instances that are live and operating normally. This means you need to be able to monitor your instances and distribute your customers to the online instances.

Services are usually reachable by a DNS address. You need to be able to control address name resolution, preferably by geography, to point your customers to the best instances. You can point directly to instances or have proxies monitor a couple of nearby instances and proxy the traffic based on policies.

There are many different kinds of policies. For example, if the proxy is able to monitor how busy an instance is, it can direct new requests to the less busier instances. Or, it can just distribute requests using a round-robin approach.

There's also another common policy that routes traffic according to the IP address. For example, if you have three instances, you use the remainder of the remainder of the devision of the IP address (as a 4-byte integer) by three to choose the instance (possible values would be 0, 1, and 2).