92 Cloud Computing
The personalized dashboard summarizes active events by business system
and severity.
Load Balancing
With the explosive growth of the Internet and its increasingly important
role in our lives, the traffic on the Internet is increasing dramatically, which
has been growing at over 100% annualy. The workload on servers is increas-
ing rapidly, so servers may easily be overloaded, especially serversfor a popu-
lar web site. There are two basic solutions to the problem of overloaded
servers, One is a single-server solution, i.e., upgrade the server to a higher-
performance server. However, the new server may also soon be overloaded,
requiring another upgrade. Further, the upgrading process is complex and
the cost is high. The second solution is a multiple-server solution, i.e., build
a scalable network service system on a cluster of servers. When load
increases, you can simply add one or more new servers to the cluster, and
commodity servers have the highest performance/cost ratio. Therefore, it is
more scalable and more cost-effective to build a server cluster system for
network services.
A server is limited in how many users it can serve in a given period of
time, and once it hits that limit, the only options are to replace it with a
newer, faster machine, or to add another server and share the load between
them. A load balancer can distribute connections among two or more serv-
ers, proportionally cutting the work each has to do. Load balancing can help
with almost any kind of service, including HTTP, DNS, FTP, POP/IMAP,
and SMTP. According to the online web encyclopedia Wikipedia, load bal-
ancing is
a technique to spread work between two or more computers, net-
work links, CPUs, hard drives, or other resources, in order to get
optimal resource utilization, maximize throughput, and minimize
response time. Using multiple components with load balancing,
instead of a single component, may increase reliability through
redundancy. The balancing service is usually provided by a dedi-
cated program or hardware device (such as a multilayer switch). It is
commonly used to mediate internal communications in computer
clusters, especially high-availability clusters.
24
24. http://en.wikipedia.org/wiki/Load_balancing_(computing), retrieved 10 Feb 2009.
Chap3.fm Page 92 Friday, May 22, 2009 11:25 AM
Where Open Source Software Is Used 93
Cloud-based server farms can achieve high scalability and availability
using server load balancing. This technique makes the server farm appear to
clients as a single server. Load balancing distributes service requests from cli-
ents across a bank of servers and makes those servers appear as if it is only a
single powerful server responding to client requests.
In most common server load balancing architectures, an incoming
request is redirected to a load balancer that is transparent to the client mak-
ing the request. Based on predetermined parameters, such as availability or
current load, the load balancer typically uses a round-robin scheduling algo-
rithm to determine which server should handle the request and then for-
wards the request on to the selected server. To make the final determination,
the load balancing algorithm, the load balancer retrieves information about
the candidate server’s health and current workload in order to verify its abil-
ity to respond to a request.
Load balancing solutions can be divided into software-based load bal-
ancers and hardware-based load balancers. Hardware-based load balancers
are specialized boxes that include Application Specific Integrated Circuits
(ASICs) customized for a specific use.
25
ASICs enable high-speed forward-
ing of network traffic and are often used for transport-level load balancing,
because hardware-based load balancers are much faster than software solu-
tions. Software-based load balancers run on standard operating systems and
standard hardware components such as desktop PCs. We will look at an
open source software solution called Linux Virtual Server next.
Linux Virtual Server Load Balancer
The Linux Virtual Server is an advanced load balancing solution that can be
used to build highly scalable and highly available network services such as
HTTP, POP3, SMTP, FTP, media and caching, and Voice over Internet
Protocol (VoIP). There are more than a few open source load balancing
applications available today, but the Linux Virtual Server (LVS) continues
to be one of the most popular. LVS is a simple, powerful product used for
load balancing and fail-over. LVS behavior is controlled at runtime by issu-
ing commands using a Command Line Interface (CLI). The syntax used for
issuing these commands is very straightforward and simple. The LVS clus-
ter
26
system is also known as a load balancing server cluster. It is built over a
cluster of physical servers with the load balancer running on top of the
25. Gregor Roth, “Server Load Balancing Architectures, Part 1: Transport-Level Load Balancing:
High Scalability and Availability for Server Farms,” JavaWorld.com, 21 Oct 2008, retrieved
10 Feb 2009.
Chap3.fm Page 93 Friday, May 22, 2009 11:25 AM
94 Cloud Computing
Linux operating system. The architecture of the server cluster is fully trans-
parent to end users, and they interact with it as if they were using a single
high-performance virtual server.
The physical servers and the load balancers may be interconnected by
either a -speed local-area network (LAN) or by a geographically dispersed
wide-area network (WAN). The load balancers dispatch requests to the differ-
ent servers and make parallel services of the cluster appear as a virtual service
using a single IP address. Scalability of the system is achieved by transparently
adding or removing servers (often referred to as nodes) in the cluster. High
availability is provided by detecting node or daemon failures and reconfigur-
ing the system dynamically to prevent performance degradation.
The LVS employs a common three-tier architecture that consists of the
load balancer, which acts as a front end to the cluster and balances (or dis-
tributes) requests from clients across a set of servers (sometimes called a
bank) so that the client’s perception is that all the services come from a sin-
gle IP address. The second tier, the server cluster, is a group of servers run-
ning network services, such as HTTP, POP3, SMTP, FTP, or DNS. The
third tier is shared storage, which provides a shared storage space for the
servers so it is easy for them to use/reuse the same contents and provide the
same services for each client request.
The load balancer itself is the primary entry point of server cluster sys-
tems. It can run Internet Protocol Virtual Server (IPVS), which implements
transport-layer load balancing inside the Linux kernel. In network parlance,
this is known as Layer-4 switching. IPVS running on a host can direct
requests for Transmission Control Protocol (TCP) and User Datagram Pro-
tocol (UDP)
27
services to the physical servers, and this redirect makes it
appear as if it were a virtual service running on a single IP address. When
IPVS is used, all servers must provide the same services and content. The
load balancer forwards a new client request to a server based on a scheduling
algorithm and the current workload for each server. Regardless of which
server is selected, the client should always see the same result. The number
of nodes active in a server cluster can be changed according to the load that
system encounters. When all servers are operating at capacity, more servers
can be added to handle a larger workload. For most Internet services,
26. According to the online encyclopedia Wikipedia, “A computer cluster is a group of linked
computers, working together closely so that in many respects they form a single computer.”
http://en.wikipedia.org/wiki/Computer_cluster, retrieved 10 Feb 2009.
27. TCP and UDP are common transport protocols defined in Request for Comment RFC 1122.
Chap3.fm Page 94 Friday, May 22, 2009 11:25 AM
Where Open Source Software Is Used 95
requests are not usually interrelated and can be run in parallel on multiple
servers. As the number of nodes active in a server cluster increases, the per-
formance of the entire cluster scales linearly.
Physically, the shared storage tier can be a database system, a network
file system, or a distributed file system, or any combination thereof. The
data that cluster nodes must update dynamically should be stored in data-
base systems because whenever a node performs a read or write operation in
a database system, the database system can guarantee the integrity and con-
sistency of concurrent data access. Static data is stored using a network file
system so data can be shared by all cluster nodes. Because scalability using a
single network file system is limited, it can only support data operations
from a limited number of nodes (typically four to eight).
For large-scale cluster systems, distributed file systems
28
can be used for
shared storage and scaled according to system requirements. The load bal-
ancer, server cluster, and shared storage are usually interconnected by high-
speed networks, such as Gigabit Ethernet, so the network will not become a
bottleneck. Common methods used to construct the clusters are Domain
Name System (DNS)-based load balancing and dispatcher-based load bal-
ancing. We will take a look at both methods in the following sections.
DNS-Based Load Balancing Clusters
DNS load balancing is probably the simplest method for building a net-
work service cluster.
29
The Domain Name System is a hierarchical naming
system for computers, services, or any resource used on the Internet. DNS
associates various information with domain names assigned to such Internet
participants. DNS translates Internet domain names (which are meaningful
to humans) into the binary identifiers associated with networking equip-
ment in order to locate and address Internet devices globally. This process is
known as name resolution and is used to distribute requests to different IP
addresses of cluster servers.
When a DNS request comes to the DNS server, it provides an available
IP address based on scheduling strategies such as are used in a round-robin
scheduler. Subsequent requests from clients using the same local caching
name server are sent to the same server if they occur within the specified
time-to-live (TTL) parameter set for name resolving. The original idea of
28. A Distributed File System (DFS) provides a means for administrators to create logical views
of directories and files that can be used in a cluster regardless of where those files physi-
cally reside on the network.
29. http://www.linuxvirtualserver.org/why.html, retrieved 10 Feb 2009.
Chap3.fm Page 95 Friday, May 22, 2009 11:25 AM
96 Cloud Computing
TTL was that it would specify a time span in seconds that, when expired,
would cause the packet to be discarded. Because every router along the path
from one device to another is required to subtract at least one count from
the TTL field, the count is usually used to mean the number of router hops
the packet is allowed to take before it must be discarded. Each router that
receives a packet subtracts one from the current count in the TTL field.
When the count reaches zero, the router detecting it will discard the packet
and send an Internet Control Message Protocol (ICMP) message back to
the originating host, letting it know that the packet was discarded.
The caching nature of clients in a hierarchical DNS system can easily
lead to a dynamic load imbalance condition across cluster nodes. This
makes it difficult for a node to operate efficiently at peak load capacity.
Since it is impossible for the DNS system to guess the TTL value of a
domain name, adjustments are often required to “tweak” the system in
order for it to operate efficiently. If the TTL value is set too small, DNS
traffic increases and the DNS server itself will bottleneck. If the TTL value
is set too high, the dynamic load imbalance will only get worse. Even if the
TTL value is set to zero, scheduling granularity is based on each host, so dif-
ferent user access patterns can also lead to dynamic load imbalances. This is
because some people may pull lots of pages from a site, while others may
just surf a few pages and leave. Load imbalance can also occur when a server
node fails and the client request that was mapped to the failing IP address
responds to the client, who often exacerbates the problem by clicking the
reload or refresh button in the browser, sending yet another request to a
dead node. Other nodes in the cluster pick up the load of the failed node,
and the workload for all nodes increases accordingly.
Dispatcher-Based Load Balancing Clusters
A dispatcher performs intelligent load balancing by using server availability,
capability, workload, and other user-defined criteria to determine where to
send a TCP/IP request. The dispatcher component of a load balancer can
distribute HTTP requests among nodes in a cluster. The dispatcher distrib-
utes the load among servers in a cluster so the services of nodes appear as a
virtual service on a single IP address; end users interact as if it were a single
server, without knowing anything about the back-end infrastructure. Com-
pared to DNS-based load balancing, dispatcher load balancing can schedule
requests at a much finer granularity (such as per connection) for better load
balancing among servers. Failure can be masked when one or more nodes
fail. Server management is becoming easy with the new tools available
Chap3.fm Page 96 Friday, May 22, 2009 11:25 AM
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.