Chapter 11. FlexVPN Load Balancer
Introduction
The FlexVPN Load Balancer feature provides the ability to cluster a number of FlexVPN gateways and distributes incoming IKEv2 connection requests evenly among each of these gateways. Using IKEv2 redirects the incoming client requests are redirected to the least loaded FlexVPN gateway based on the system and cryptographic load. This allows for a scalable pay-as-you-grow architecture where VPN gateways can be added or removed from the cluster, depending on the number of sessions required.
Components of the FlexVPN Load Balancer
The FlexVPN load balancer is constructed of a number of components. By understanding those components, you will have a great understanding of the overarching feature.
IKEv2 Redirect
RFC 5685 “Redirect Mechanism for the Internet Key Exchange Protocol Version 2 (IKEv2)” describes a method to redirect an IKEv2 client from one IKEv2 gateway to another IKEv2 gateway.
This behavior allows IKEv2 clients to connect to a single IP address or FQDN as the VPN headend and then allows the VPN gateway owning this IP address to redirect the client to another gateway to provide load balancing functionality.
An IKEv2 client will indicate the ability to perform the IKEv2 redirect mechanism and its willingness to be redirected by including a REDIRECT_SUPPORTED notification message within the IKE_SA_INIT exchange. On receipt of this indication, the IKEv2 gateway can then redirect the IKEv2 client to another IKEv2 gateway of its choosing.
The redirection is performed, using the REDIRECT notification message. This notification includes information about the IKEv2 gateway used to redirect the client to and the nonce data received from the IKEv2 client in the IKE_SA_INIT message. The redirection can be performed by the IKEv2 gateway in the IKE_SA_INIT reply message or in the IKE_AUTH message.
If the redirect is performed within the IKE_SA_INIT exchange, then the computationally expensive Diffie-Hellman key generation is not performed, however no authentication is performed by the IKEv2 gateway with regard to the client. If the redirect is performed in the IKE_AUTH exchange, then both peers authenticate each other prior to performing the redirect. This allows the IKEv2 gateway to ensure that only legitimate clients are redirected and that the client can validate it has been redirected from the legitimate gateway. However, this comes with the cryptographically expensive operations involved with authentication.
Figure 11-1 illustrates the IKE exchange when a redirect occurs within the IKE_SA_INIT exchange.
Figure 11-2 illustrates the IKE exchange when a redirect occurs within the IKE_AUTH exchange.
The inclusion of the nonce data within the REDIRECT notification ensures that an unsolicited forged reply cannot be used by an attacker to redirect clients to IKEv2 gateways of their choosing. because the nonce data is chosen randomly, an out-of-band attacker would be required to guess this value making this attack computationally infeasible.
When a client receives the REDIRECT notification, assuming that the nonce value matched the one that was sent originally the client will have all the information required to connect to the new IKEv2 gateway. This exchange will then occur; however, a REDIRECTED_FROM notify payload will be included which contains the IP address of the gateway that redirected the client.
Figure 11-3 illustrates the IKE_SA_INIT exchange once a client has been redirected from a gateway.
Hot Standby Routing Protocol
The role of the hot standby routing protocol (HSRP) is to make a group of layer 2 adjacent routers appear as a single virtual router. This is achieved by using a virtual IP address (VIP), which is an IP address that belongs to the HSRP group and is owned by a single physical router at any given time. This physical router, known as the active router, will receive all traffic sent to the VIP and send traffic with the VIP as the source IP address. The VIP has a related virtual MAC address. The other physical routers, known as standby routers, are not actively passing traffic but maintain HSRP state with the active router. When the active router fails, a standby router automatically takes over the active role and begins sending and receiving traffic, using the VIP. All routers in the HSRP group must have an IP address on the local layer 2 network that is separate from the VIP.
Note
HSRP is documented in RFC 2281 “Cisco Hot Standby Router Protocol (HSRP).”
The mechanism used to select the master is achieved using a priority value, which is user configurable and allows for the value between 1 and 255, with the higher value being preferred. In the case of two routers having the same priority, the numerical IP address would be the differentiator.
The following example illustrates HSRP configured on two routers using the VIP of 10.10.10.100 with a priority of 255 for router1 and 200 for router2 (the default being 100). The standby group name is defined as “group1”. With this configuration, router1 will be the HSRP master, and router2 will be the HSRP standby.
Router1#show run interface Ethernet0/0
interface Ethernet0/0
ip address 10.10.10.1 255.255.255.0
standby 1 ip 10.10.10.100
standby 1 priority 255
standby 1 name group1
Router2#show run interface Ethernet0/0
interface Ethernet0/0
ip address 10.10.10.2 255.255.255.0
standby 1 ip 10.10.10.100
standby 1 priority 200
standby 1 name group1
FlexVPN IKEv2 Load Balancer
FlexVPN IKEv2 Load Balancer introduces the ability to cluster a number of FlexVPN servers together to terminate IKEv2 SAs from FlexVPN clients. Clients initially connect to a master server and are redirected to the least-loaded server within the cluster, using the IKEv2 redirection mechanism as described in RFC 5685 “Redirect Mechanism for the Internet Key Exchange Protocol Version 2 (IKEv2).”
FlexVPN Load Balancer uses the concept of a cluster of VPN servers. These servers must be in the same layer 2 network and be part of the same HSRP group. The HSRP active router will also be the master of the cluster, owning the virtual IP address (VIP) for the cluster, with other routers in the HSRP group in standby state as cluster members (also known as slaves).
All members in the cluster communicate their load to the cluster master, the cluster master maintains a record of all cluster members and their load, which is then used when calculating the least-loaded member.
When a client connects to HSRP VIP, which is the cluster master, the master will redirect the client to the physical address of the least-loaded cluster member. The client will then initiate a new IKEv2 session directly to the cluster member that it was redirected to. This behavior is illustrated in Figure 11-4.
Configuration on the client is simple when the clients connect only to the IP address or FQDN of the HSRP VIP, resulting in minimal room for configuration error. The ability to add new cluster members allows the cluster to be in a pay-as-you-grow fashion. A cluster can be built with a small number of members (or even only one), with then new hardware added as required. The hardware and software of devices do not have to match, resulting in the ability to add hardware with enough capacity to service the required additional clients and to upgrade cluster members without having to bring the whole cluster down.
Because the load is evenly distributed between members, assuming that a load is evenly distributed between cluster members and a new member is added to the cluster, the new member will receive all new requests until its load matches all other members.
To configure the IKEv2 cluster feature, the crypto ikev2 cluster command is used to enter the cluster load management feature. The sub-menu contains attributes that can be used to customize the cluster. The following example illustrates how to configure the feature.
Router(config)#crypto ikev2 cluster
Router(config-ikev2-cluster)#
There are three mandatory components to this feature.
The HSRP group must be configured, using the standby-group group_name command.
The maximum number of IPsec and IKEv2 SAs must be configured by using the slave max-session number command. The number is configurable between 1 and 100,000.
Once the cluster feature has been configured, IKEv2 clustering must be enabled using the no shutdown command within the IKEv2 cluster. This will then activate the cluster feature.
Note
Before this command is enabled, the cluster will show as shut down:
Router#show crypto ikev2 cluster
Cluster is shutdown
The following sections describe specific configuration attributes that relate to the IKEv2 cluster feature. For this feature to be active, the cluster member must also enable the sending of the IKEv2 Redirect.
Cluster Load
The FlexVPN cluster master maintains a status of the load on all other cluster members. Cluster members connect to the master, using a Cisco propriety protocol called Cisco load balancing (CLB) protocol. CLB operates over TCP port 2012 and is used by cluster members to inform the master of their state (enabling debug crypto ikev2 cluster detail will display details of the load being sent from cluster members to the cluster master). Cluster members send details of their load at 3000 ms intervals; the cluster member will also send a hello packet every 1000 ms, which will be acknowledged by the master. There is a default hold time of 3000 ms. Because the hello time is set to 1000 ms, three missed hellos will be required to occur before the sessions are pulled down. This ensures that the session between the cluster and the master is not torn down should one or two hello packets be missed for reasons, such as congestion or network re-convergence.
The command show crypto ikev2 cluster on the cluster master will show the status of the load on cluster members. The least-loaded cluster member will be denoted with a star (*) next to it, and it will have the next IKEv2 session redirect to it.
The following example illustrates how to view the status of an IKEv2 cluster master. We see 10.10.10.3 is the least-loaded member of the cluster and will have the next IKEv2 session redirected to it.
Router#show crypto ikev2 cluster
Role : CLB Master
Status : Up
CLB Slaves : 2
Cluster IP : 10.10.10.100
Hold time : 3000 msec
Overload limit : 90%
Codes : '*' Least loaded, '-' Overloaded
Load statistics:
Gateway Role Last-seen Prio Load IKE Ipsec
--------------------------------------------------------------
-10.10.10.2 Master -- 100 90.0% 1 2
*10.10.10.3 Slave 00:00.728 100 13.2% 0 0
10.10.10.1 Slave 00:00.766 100 28.2% 1 2
To configure the port used and the timers, adjust the following parameters under the ikev2 cluster:
holdtime 100–120000
The hold time interval is in milliseconds. The default is 3000 ms.
port 1–65535
TCP port for CLB. The default is 2012.
slave hello 100–30000
Slave hello time in milliseconds. The default is 1000 ms.
slave max-session 1–100000
Maximum number of IKEv2 and IPsec Security Associations. The default is 10.
slave update 100–60000
Load update time in milliseconds. The default is 3000 ms.
slave priority 1–100
Slave priority is a value set on the cluster member that influences the load calculation. The higher the slave priority, the lower is the load calculated on the cluster member. The following example shows Router1 as a member of the cluster, which has an initial slave priority of 100 and a load of 13.2%. After amending the slave priority to 50 on Router1, we see the load increase to 19.8%.
Router1(config)#crypto ikev2 cluster
Router1(config-ikev2-cluster)#do show crypto ikev2 cluster
Role : CLB Slave
Status : Up
Cluster IP : 10.10.10.100
Hold time : 3000 msec
Hello-interval : 1000 msec
Update-interval: 3000 msec
Load statistics:
Gateway Last-Ack Prio Load IKE IPsec
-------------------------------------------------------
10.10.10.3 00:00.725 100 13.2% 0 0
Router1(config-ikev2-cluster)#slave priority 50
Router1(config-ikev2-cluster)#do show crypto ikev2 cluster
Role : CLB Slave
Status : Up
Cluster IP : 10.10.10.100
Hold time : 3000 msec
Hello-interval : 1000 msec
Update-interval: 3000 msec
Load statistics:
Gateway Last-Ack Prio Load IKE IPsec
-------------------------------------------------------
10.10.10.3 00:00.865 50 19.8% 0 0
The ability to amend the load can be used to amend the configurable values to ensure that the master unit is seen as the most loaded member in the cluster. This can be useful to ensure that the master unit is not overloaded with IKEv2 sessions, so it has capacity to reply to redirect clients to other cluster members and also service any clients that do not support the redirect feature.
The standby group is used to tie the HSRP group to the cluster.
The load of cluster members is calculated using the following parameters:
System load—Memory percentage used
System load—Average CPU usage for past 5 seconds
Crypto load—IPsec Security Associations
Crypto load—IKEv2 SAs
Overload limit
Slave Priority
The system and crypto load parameters can be configured to have less precedence when calculating the percentage of load on a cluster member. By default, the weight is set to 100; however, this can be lowered, which will result in the load being calculated to a lower value. This is beneficial in circumstances in which different hardware is used for cluster members.
The following example illustrates the IKEv2 cluster weight configuration.
crypto ikev2 cluster
master weight crypto-load 100
master weight system-load 100
The overload limit is the load limit that is calculated locally for the cluster member. Should a cluster member reach this limit, it will be marked as unavailable and will not accept any more connections. The default is set to 90%.
The following example illustrates the IKEv2 cluster overload-limit configuration.
crypto ikev2 cluster
master overload-limit 90
IKEv2 Redirect
The ability to redirect a FlexVPN client to another VPN gateway is based on the standard described in RFC 5685 Redirect Mechanism for the Internet Key Exchange Protocol Version 2 (IKEv2).
For clients to have the ability to use the IKEv2 redirect mechanism, the client must explicitly specify that it supports IKEv2 redirects. This is achieved by including a REDIRECT_SUPPORTED payload, which is sent in the initial IKE_SA_INIT message. This can be configured on the client by the command crypto ikev2 client redirect. If the IKE_SA_INIT request did not indicate support for the redirect mechanism, the responder must not send the REDIRECT payload to the VPN client.
The cluster master can send the IKEv2 redirect in either the IKE_SA_INIT or in the IKE_AUTH exchange. Sending the IKEv2 redirect in the encrypted IKE_AUTH exchange offers the ability to redirect only authenticated clients because the redirect will occur after both parties have performed a mutual authentication.
On the cluster member, the global command to configure sending redirects is crypto ikev2 redirect gateway init to enable sending of redirects in IKE_SA_INIT or crypto ikev2 redirect gateway auth to send a redirect in IKE_AUTH.
If the ability to send IKEv2 redirects is enabled when the cluster feature is not enabled, a warning message is generated as illustrated in the following example.
Router(config)#crypto ikev2 redirect gateway init
! (IKEv2 Cluster load-balancer is not enabled)
Within the IKEv2 profile, the ability to send an IKEv2 redirect can be configured, using the following command:
crypto ikev2 profile default
redirect gateway auth | init
Note that Cisco AnyConnect explicitly contains the IKEv2 redirect capability.
Once the cluster master has sent the IKEv2 redirect, the client will then initiate a new connection to the peer that was indicated in the reply from the cluster master. Within the new IKE_SA_INIT exchange a REDIRECTED_FROM payload will be included, which contains the IP address of the gateway that sent the redirect. In the case of IKEv2 load balancer, this will be the VIP. This payload also serves the purpose of indicating support for the redirect mechanism to the new VPN gateway after the client has received a redirect.
When a client is redirected to another IKEv2 peer, the initial IKEv2 SA that was used by the redirecting gateway will be deleted. This can be seen in the following example when the client receives the responder’s IKE_SA_INIT containing the redirect; the original SA is then deleted.
IKEv2 IKE_SA_INIT Exchange RESPONSE
Payload contents:
NOTIFY(REDIRECT)
IKEv2:(SA ID = 1):Processing IKE_SA_INIT message
IKEv2:(SA ID = 1):Redirect acceptance check with platform
IKEv2:(SA ID = 1):Redirect accepted, clean-up IKEv2 SA, platform will initiate
new request
IKEv2:(SA ID = 1):
IKEv2:(SA ID = 1):Failed SA init exchange
IKEv2:(SA ID = 1):Initial exchange failed
IKEv2:(SA ID = 1):Initial exchange failed
IKEv2:(SA ID = 1):Abort exchange
IKEv2:(SA ID = 1):Deleting SA
Redirect Loops
A client request could be redirected multiple times in a sequence because of either an incorrect configuration or a denial-of-service (DoS) attack. If a client were redirected continuously between gateways, the client would never establish a session with its intended peer, and resources on the gateways redirecting the client could be consumed resulting in loss of service.
To prevent the possibility of clients being redirected in loops, a client can be configured, using the crypto ikev2 redirect client command with the max-redirects keyword argument pair, to not accept more than a specific number of redirects for a particular IKEv2 SA setup.
The value that can be configured is between 1 and 255 with a default of 5. Should a client meet this limit, it will abort the connection attempt.
FlexVPN Client
For a FlexVPN client to access an IKEv2 load balancer solution, the client must be configured with the crypto ikev2 redirect client command that enables the client to advertise the ability to use IKEv2 redirect and also to act on it.
Additionally, the tunnel interface configured on the client must not have a static destination address. The destination address of the tunnel must be set to dynamic because the client will first connect to the VIP and then be redirected to the final destination address. If the tunnel is configured with a static destination address, the client will not act on the IKEv2 redirect.
To enable a FlexVPN client to use a dynamic address, the client should be configured to use a tunnel interface with a dynamic destination. The following example illustrates a FlexVPN client configuration with a dynamic destination address.
crypto ikev2 client flexvpn flex_client
peer 1 10.10.10.1
client connect Tunnel1
interface Tunnel1
ip address negotiated
tunnel source Ethernet0/0
tunnel destination dynamic
tunnel protection ipsec profile default
Note
The Cisco AnyConnect client supports the ability to use IKEv2 redirect and can integrate with the IKEv2 cluster.
Troubleshooting IKEv2 Load Balancing
Should you experience issues with the IKEv2 load-balancing feature the command show crypto ikev2 cluster will detail the role of the cluster member (Master or Slave) along with details of the load of the device. As previously mentioned, this command issued on the cluster master will display the least-loaded member, which will be the next device redirected to.
The following example illustrates the show standby command that will display details of the HSRP configuration and status.
Router#show standby
Ethernet0/0 - Group 1
State is Listen
Virtual IP address is 10.10.10.100
Active virtual MAC address is 0000.0c07.ac01
Local virtual MAC address is 0000.0c07.ac01 (v1 default)
Hello time 3 sec, hold time 10 sec
Preemption disabled
Active router is 10.10.10.2, priority 255 (expires in 9.120 sec)
Standby router is 10.10.10.3, priority 100 (expires in 10.816 sec)
Priority 100 (default 100)
Group name is "group1" (cfgd)
Debugging can be enabled specifically for the ikev2 cluster feature by using the debug crypto ikev2 cluster command; this will enable specific debug information pertaining to the IKEv2 cluster such as received or sent messages.
The debug crypto ikev2 cluster detail command will enable detailed debugging, which will create a very verbose output, containing detailed information about messages and contents and load information calculated for all update messages received. It should be noted that as the output is quite verbose, caution should be exercised to prevent overloading of lower-end routers.
The debug crypto ikev2 cluster error will enable debugging for errors encountered within the IKEv2 cluster feature.
Standard IKEv2 debug commands such as debug crypto ikev2 and debug crypto ikev2 packet hexdump can be used to verify the REDIRECT payloads being included within the IKEv2 exchange.
The following example illustrates when the debug crypto ikev2 packet hexdump command was enabled on a client. After redirection, we see the client initiate a connection including the REDIRECT_FROM payload, and we can confirm that 0A 0A 0A 64 is the HEX for 10.10.10.100 and thus can determine where the REDIRECTED_FROM payload originated from
NOTIFY(REDIRECTED_FROM) Next payload: NONE, reserved: 0x0, length: 14
Security protocol id: Unknown - 0, spi size: 0, type: REDIRECTED_FROM
01 04 0A 0A 0A 64
IKEv2 Load Balancer Example
In this section, we will describe a typical IKEv2 load balancer deployment. Figure 11-5 illustrates the IKEv2 load balancer in action. Three routers (Router1, Router2, and Router3) are configured as a cluster, with a number of clients connected to these.
The WAN facing HSRP configuration uses the IP address 10.10.10.100 for the VIP. Each router also has a unique IP address configured with Router1 having 10.10.10.1, Router2 having 10.10.10.2, and Router3 having 10.10.10.3.
The router northbound of the load balancer must be aware of the address pools that are configured to allocate clients IP addresses on the cluster. This allows for clients that access northbound data to then allow the returning traffic back to the correct cluster member, which is then encapsulated and sent back to the client.
The following configuration example from Router1 illustrates the configuration required to enable the IKEv2 cluster feature along with the necessary IKEv2 configuration to enable remote access clients to connect.
Note
The configuration for interfaces and the certificate authority and certificate map have been removed.
HSRP is configured on the WAN facing interface with a VIP of 10.10.10.100 and a name of “group1.” For all other routers within the cluster, the IP address will be unique. The VIP and group name match what is configured in the following example.
interface Ethernet0/1
ip address 10.10.10.1 255.255.255.0
standby 1 ip 10.10.10.100
standby 1 name group1
IKEv2 redirects are enabled by using the crypto ikev2 redirect gateway init command with clients redirected in the IKE_SA_INIT exchange.
The following example shows the IKEv2 cluster feature configuration, with the HSRP group name configured in the previous example.
crypto ikev2 cluster
standby-group group1
slave max-session 10
no shutdown
The following example shows the IKEv2 configuration that is required to allow remote clients to connect. The local pool configuration is unique to this router, and other cluster members will have a different range configured with the pool. The rest of the configuration will be used across all other cluster members.
crypto ikev2 profile default
match certificate certmap
identity local dn
authentication remote rsa-sig
authentication local rsa-sig
pki trustpoint 1
aaa authorization group cert list default default
virtual-template 1
crypto ikev2 authorization policy default
pool pool1
route set interface
interface Virtual-Template1 type tunnel
ip unnumbered Loopback1
tunnel source Ethernet0/1
tunnel protection ipsec profile default
ip local pool pool1 172.16.1.1 172.16.1.255
The following example illustrates the master (Router1) that has a single client session connected to it. The next IKEv2 session to be received on VIP of the master will be redirected to Router3 (10.10.10.3).
Router1#show crypto ikev2 cluster
Role : CLB Master
Status : Up
CLB Slaves : 2
Cluster IP : 10.10.10.1
Hold time : 3000 msec
Overload limit : 90%
Codes : '*' Least loaded, '-' Overloaded
Load statistics:
Gateway Role Last-seen Prio Load IKE IPsec
--------------------------------------------------------------
10.10.10.1 Master -- 100 31.7% 1 2
*10.10.10.3 Slave 00:00.224 100 16.2% 0 0
10.10.10.2 Slave 00:00.600 100 16.2% 0 0
The following example illustrates the syslog messages, which were generated using debug crypto ikev2 on the master, and shows a client connecting to the VIP and being redirected to Router3. In the IKE_SA_INIT request, we can see that the REDIRECT_SUPPORTED notify payload is included.
IKEv2:Received Packet [From 10.10.20.6:500/To 10.10.10.1:500/VRF i0:f0]
Initiator SPI : 5DF2CD9E0A2E1171 - Responder SPI : 0000000000000000 Message id: 0
IKEv2 IKE_SA_INIT Exchange REQUEST
Payload contents:
SA KE N VID VID NOTIFY(NAT_DETECTION_SOURCE_IP)
NOTIFY(NAT_DETECTION_DESTINATION_IP) NOTIFY(REDIRECT_SUPPORTED)
The master will then redirect the client to the least-loaded gateway (10.10.10.3) by sending a reply containing a redirect notify with this information.
IKEv2:(SESSION ID = 10,SA ID = 1):Redirect check with platform for load-balancing
IKEv2:(SA ID = 1):IKEv2 SA got Redirected to '10.10.10.3'
IKEv2:(SESSION ID = 10,SA ID = 1):Sending Redirect notification
IKEv2:(SESSION ID = 10,SA ID = 1):Sending Packet [To 10.10.20.6:500/From 10.10.10.1:500/VRF i0:f0]
Initiator SPI : 5DF2CD9E0A2E1171 - Responder SPI : 0000000000000000 Message id: 0
IKEv2 IKE_SA_INIT Exchange RESPONSE
Payload contents:
NOTIFY(REDIRECT)
The client will then attempt to establish an IKEv2 SA with 10.10.10.3.
Summary
The FlexVPN load balancer feature combines HSRP and the IKEv2 redirect feature to provide a scalable IKEv2 VPN headend solution. Additional hardware can be added to the headend cluster, allowing for a pay-as-you-grow model where new capacity can be easily added when required.
Because the IKEv2 redirect feature is standardized, any compatible third-party VPN client based on IKEv2 can be used with the FlexVPN load balancer. Clients connect to the HSRP VIP and are then redirected to the least-loaded gateway. This allows for an equal load to be distributed across cluster members.