Chapter 1. Introduction to IPsec VPNs

This chapter introduces the concepts of Virtual Private Networks (VPNs), focusing specifically on Internet Protocol Security (IPsec); this chapter will also introduce the Internet Key Exchange protocol version 2 (IKEv2).

Ten years ago IPsec VPNs were commonly employed only for connectivity between site locations or for remote mobile workers requiring secure connectivity from hotels, where specialist software was required. With the advance of the Internet of Things, the number of devices that require secure connectivity over insecure networks is almost endless; the ability to connect, communicate with, and remotely manage every IP-enabled device over a secure medium has increased the demand for VPN technologies. No longer can devices be configured to use insecure protocols across the Internet; the mandatory method of communication requires security, and for this reason the use of IPsec VPNs has increased in recent years, with demand expected to continue rising.

This chapter describes the components that are used to create IPsec VPNs and will make the connection between Security Protocols and the IPsec Security Associations. Security services provided by IPsec are described, along with how the components of IPsec operate, with specific focus on the Security Association Database, Security Policy Database, and Peer Authorization Database.

The cryptographic components of IKEv2 are introduced to provide an understanding of what is happening internally when creating a secure authenticated IPsec Security Association. Asymmetric cryptography is described, as well as the way it is used within the establishment of an IPsec Security Association. Authentication using digital signatures, and shared secrets is covered, along with the construction of shared secret key material. Symmetric cryptography is described and how this relates to both Authentication Header and Encapsulation Security Payload and the services that these provide.

Both modes of IPsec are discussed, tunnel mode and transport mode, and the benefits that these bring.

When IPsec VPNs are used, traffic will be protected to ensure that an observer cannot view the plaintext data; this is achieved by encryption that provides confidentiality. An intruder cannot tamper with the data. This is achieved by cryptographic hashing and signing of the data exchanged, which provides integrity. A VPN must be established only with a chosen peer; this is achieved using mutual authentication.

Please be aware of the differences between a VPN using IPsec and a VPN using Multiprotocol Label Switching (MPLS). MPLS uses labels to differentiate traffic sent on the wire; this is a method to separate traffic, but unlike IPsec, it offers no confidentiality or integrity protection.

At the time of writing, almost every operating system ships with an IPsec VPN client, and there are countless hardware devices that provide various forms of IPsec VPN gateway functionality. IPsec VPNs are a popular choice now for secure connectivity over the Internet or for delivering secure communications over untrusted networks, such as the Internet.

Internet Key Exchange (IKE) creates one or more pairs of IPsec Security Associations (SA). The SAs are what data will be sent within; data is encapsulated and encrypted prior to being sent onto the wire. The cryptographic key material that is used to protect the IPsec Security Association is negotiated in the IKE exchange.

The IPsec Security Association has a defined lifetime. It will expire after a set duration, volume of data, or both.

IPsec establishes tunnels between endpoints; these can also be described as peers. The tunnel can be protected by various means, such as integrity protection and/or confidentiality protection.

The following sections describe at a high level each of the building blocks that are used to construct and maintain IPsec sessions.

Both protocols support two modes of use: transport mode and tunnel mode. When transport mode is used, both Authentication Header and Encapsulation Security Payload provide protection primarily for protocols that are sent by the IPsec peer. Originally this was a host device, but more recently it can be a network device also. When tunnel mode is used, Authentication Header and Encapsulation Security Payload are applied to tunnelled IP packets.

The SA is protected by one of the security protocols described above.

Each SA is identified by a Security Parameter Index (SPI). This is a unique 32-bit value that is included in the security protocol header.

IPsec security services require the use of cryptographic keys, which are negotiated using a separate function that is achieved by the Internet Key Exchange.

The majority of the security services are provided through use of either of the two traffic security protocols: Authentication Header and Encapsulating Security Payload.

IPsec creates a boundary between unprotected and protected interfaces for a host or a network device. Traffic traversing this boundary is subject to enforcement of the security policy. The security policy indicates whether packets cross the boundary without intervention, are afforded security services via Authentication Header or Encapsulation Security Payload, or are discarded.

It should be noted that because IPsec is an open standard, these services can be achieved in an interoperable manner.

The SPD security policy consists of a processing action; this action can be “protect,” “discard,” or “bypass.” Usually management traffic, such as IKE, would have the “bypass” action, and data traffic would have the “protect” action. The security parameters for the “protect” action include the security protocol (Encapsulation Security Payload or Authentication Header), the protocol mode (transport or tunnel), cryptographic algorithms, local and remote IPsec peers, and the granularity of the SA, based on whether the SA traffic selectors are derived from the parent SPD entry or from the packet that matched the SPD (when each packet flow creates an IPsec Security Association).

The SPD is an ordered first-match database of SPD entries in which each SPD entry defines the security policy for a traffic flow. An IPsec peer can have one or more instances of an SPD.

For plaintext outbound traffic, after processing by the SAD, the SPD is searched to determine if the packet should be protected.

For inbound traffic (protected traffic received), after the SA is determined via the SAD, the SPD is searched to check if the decrypted/authenticated packet was supposed to be protected.

An IPsec peer has an SAD instance per SPD instance.

For outbound traffic, the SAD is searched to see how the packet should be protected.

Within the IKE exchange, both peers negotiate cryptographic algorithms that are later used to protect the session and exchange Diffie-Hellman public key values; from this exchange a shared secret is derived, which is then used as the input to a function to generate key material used to protect subsequent IKE exchanges.

Four cryptographic algorithms are negotiated: an encryption algorithm, an integrity algorithm, a Diffie-Hellman public value/group, and a pseudorandom function (PRF).

The PRF is a method to generate a value that appears random from a given input. It is used for the construction of key material used for all of the cryptographic algorithms in both the IKE Security Associations and IPsec Security Associations and also some methods of authentication.

After the session has been secured, both parties must authenticate themselves. This can be achieved using either EAP (IKEv2 only), shared-secrets, or digital signatures. The methods for using EAP and shared secrets are covered in chapter 2, “IKEv2: The Protocol.”

IKEv2 uses both asymmetric and symmetric cryptography.

This process of transformation is called encryption or encipherment. The reverse process, during which ciphertext is transformed back into plaintext, is called decryption or decipherment.

The process requires that each party has access to the secret key. This secret key needs to be known to both sides and is required to be kept secret. Encryption algorithms that have this property are called symmetric cryptosystems or secret key cryptosystems.

Each key will perform the inverse of the other. For example, a private key is used to encrypt plaintext or to create a digital signature, whereas the associated public key is used in the opposite operation, in these examples to decrypt ciphertext or to verify a digital signature. When digital signatures are used, one peer will sign a known value with its private key, and the opposing peer will validate this with the cryptographically linked public key, which is included in the certificate of the peer that is signing with the private key.

When an IKE or IPsec Security Association is created, the key used as the input to the encryption function is known as SK_e (the letter “e” indicating encryption). The same key is used as input to the decryption function by the opposing peer, which aligns with the logic behind symmetric encryption, where a single key is used for both encryption and decryption. The bulk of data is encrypted using a symmetric algorithm because asymmetric encryption is computationally expensive compared to symmetric. Symmetric algorithms can be easily accelerated by hardware, unlike asymmetric cryptographic algorithms, which can be accelerated but commonly are not.

The Diffie-Hellman exchange is a form of asymmetric cryptography; the details are covered within the next section. It is important to note that data is not encrypted or signed by a public or private key, but rather a shared secret is derived between two parties using large prime numbers. However, this is still considered to be public key cryptography.

The Diffie-Hellman exchange is susceptible to a man-in-the-middle attack, where malicious observers can substitute their own values and create two separate secure channels with each party. For this reason, when IPsec VPNs are used, an additional authentication phase occurs to ensure that a man-in-the-middle attack is not possible.

The idea of the Diffie-Hellman exchange was first presented to the public in a paper by Whitfield Diffie and Martin Hellman entitled “New Directions in Cryptography” in 1976. However mathematicians in the United Kingdom’s GCHQ had developed the same concept earlier, although this research at the time was classified and not publicly released.

The following illustrates a typical Diffie-Hellman exchange, the public value being denoted by ‘A’ and ‘B’; these values are very small and are used as an example.

p = 23

g = 5

p and g are constants defined by the Diffie-Hellman group in use; the value can be seen in RFCs that describe the Diffie-Hellman parameters. The value p is a prime and g is the generator. The reason that these values are defined and not generated by each peer is that they are used as a constant in the value of the shared secret. Each peer selects only a local public and private value. As the value p is used as a modulus, in this example it gives 23 possible values. Standard implementations use the value of p in 100s of digits, which makes them computationally infeasible to reverse engineer. This is known as the discreet logarithm problem. The value of 2 for the generator is used because the computation is efficient on a binary-based machine and it results in a uniform distribution.

The peers, otherwise known as Alice and Bob, calculate a private value.

Alice calculates her private value to be “a”; Alice then calculates her public value “A” using the formula

A = g^a mod p

In this instance Alice calculates a = 6,

A = 5⁶ mod 23

A = 15625 mod 23

A = 8

Alice sends 8 as her public value to her peer, Bob.

Note that for a malicious observer to derive Alice’s secret value, they would have to perform all calculations until they too landed with the public value of 8, which would reveal the secret value of 6. When large numbers are used, this is computationally impossible, even with cloud-based computing.

Bob calculates his private value “b” as 15 and then calculates his public value “B” as

B = g^b mod p

B = 5¹⁵ mod 23

B = 30,517,578,125 mod 23

B = 19

Bob then sends his public value 19 to Alice.

Table 1-1 illustrates the generation of the shared secret by both Alice and Bob. Both parties derive the same shared secret.

It can be seen that (g^a mod p)^b mod p equals (g^b mod p)^a mod p equals =g^ab mod p.

Both Alice and Bob have arrived at the same shared secret; however, anyone monitoring the network exchange will have seen only the public values that were transmitted, so as long as the private values stay secret, the exchange will be computationally secure. The phrase “computationally secure” is used to indicate where the best possible algorithm for obtaining the private value from the public value requires X operations, where X is such a large number that it is computationally infeasible to perform this many operations by an attacker.

Diffie-Hellman can also be used with an elliptic curve, known as ECDH (Elliptic Curve Diffie Hellman). In an ECDH exchange, points on an elliptic curve are sent between parties in a manner similar to the example above. If ECDH is used, a known elliptic curve would be defined, such as;

Y² = X³ + X – 3

A base point (G) on the curve will be agreed to before the exchange, for example G = (1,76) denoting the X and Y coordinates on the elliptic curve. Each peer calculates a private value (the peers are referred to as Alice and Bob).

Alice selects a private value a, which is used with the base point G. In this case a = 23, and the result is a coordinate on the elliptic curve, A = (2,150), which is sent to Bob.

Bob selects a private value b, and using the base point G, he obtains the coordinates of his public value, B (123,187), which is sent to Alice.

Alice can then use her private value (23) and Bob’s public coordinates (2,150) to obtain a shared point on the elliptic curve. Likewise, Bob can use his private value (86) and Alice’s public coordinates (2,150) to obtain a shared point on the elliptic curve.

Alice’s computation = [a]B = [23](123,187) = (156, 75)

Bob’s computation = [b]A = [86](2,150) = (156, 75)

The shared secret is then taken to be the x-coordinate (in this case the value 156) of the computed point.

When ECDH is used instead of modulo Diffie-Hellman, the size of the exchange attribute will be considerably smaller for ECDH. Not only are the sizes smaller, the cryptographic workrate is considerably less too. You get more effective use of your resources with regards to the size of key compared to cryptographic security, so for this reason (at the time of this writing) Cisco recommends, whenever it is possible, the use of Next Generation Encryption, which would use Diffie-Hellman Groups 19, 20, or 21.

The hardness of the Diffie-Hellman exchange is based on discrete logarithm problem, whereby obtaining the private key used from only knowing the public value is defined.

The following example shows an IKEv2 key exchange payload from an SA_INIT exchange; this uses the smallest Diffie-Hellman group (Group 1). Although the key exchange data (public key) looks large, it is in cryptographic terms rather weak, and the use of anything below Diffie-Hellman Group 5 is discouraged, because the small key sizes are not sufficient in light of today’s computational processing power.

Click here to view code image

    Type Payload: Key Exchange (34)
       Next payload: Nonce (40)
        0... .... = Critical Bit: Not Critical
        Payload length: 104
        DH Group #: Default 768-bit MODP group (1)
        Key Exchange Data:
dcfae25205d1afcbc1b9d9a77464ad375229a747fc48e56621608e7533d617ce2a4b4f54ff3702f
0632c17ac7e8fa8ee1c52e4d0825bd5ff1d26e0197b106dae63bff8183fab5fcf9fe2bbe5ab3fbb
a37f26be4cda595139e894fb01d5d6f6c5

The following RFCs detail the use of various Diffie-Hellman groups, including the parameters to be used:

RFC3526 More Modular Exponential (MODP) Diffie-Hellman groups for Internet Key Exchange (IKE)

RFC5114 Additional Diffie-Hellman Groups for Use with IETF Standards.

A PKI allows hosts to communicate securely by exchanging certificates and authenticating themselves to each other. The benefit of using a PKI over a shared secret is that anyone or anything within a PKI can communicate without knowing the peers’ shared secret. This is contrary to symmetric encryption.

The following components are used within a PKI.

Public-key cryptography, also called asymmetric cryptography, is a form of cryptography based on algorithms that require two separate keys, one of which is secret (or private) and one of which is public. Both the public and private keys are mathematically linked, which gives the ability to prove what is done with one key can be undone with the other.

Public key cryptography has many advantages over symmetric key cryptography: the primary one being that both parties are not required to know each other before communication can take place; instead, they all trust a common entity that signs certificates (containing an identity and public key), which is known as a chain of trust. The use of digital signatures enables the holder of a certificate and associated private key to prove their authenticity.

To validate the digital signature of the certificate authority, the receiver must first know the CA’s public key. Typically this process is handled out of band or through an operation that is performed at installation. On Cisco IOS, you must first configure a trust point, which is a cryptographic container; you can then authenticate the certificate authority. This process involves obtaining a copy of the CA’s certificate and validating the fingerprint (or signature) of it.

ECDSA signatures are smaller than RSA signatures, and ECDSA keys require less bandwidth; the use of ECDSA-sig also has advantages of computational speed and efficiency when compared to RSA-sig.

The actual method by which authentication is proven has a peer signing a block of data (called signed octets) that contains data such as the IKE identity, nonce (number used once), and a value that is derived from the SKEY (SK_pr or SK_pi). The nonce is a random value that is used as an input into cryptographic calculations to add randomness.

As the IKE identity and nonce are included in the generation of key material, this will keep an malicious observer from performing a man-in-the-middle (MITM) attack. In an MITM attack, an attacker creates a secure channel to each peer impersonating the true peer. As the signed octets contain values such as the nonce, it is computationally infeasible for attackers to guess this value prior to attempting to insert themselves into the exchange.

This block of data, which can be referred as a message, is signed by the private key, which is sent within the authentication payload described in detail in the next chapter.

The signature can be described as;

signature = sign(message, private_key)

This signature can be validated by the receiving peer, by using the public key corresponding to the private key, which has a cryptographic relationship between both:

verify (message, signature, public_key) = pass | fail

The receiving host can verify that the message was signed by the holder of the private key by verifying the data contained in the AUTH payload. This is the case when RSA-sig or ECDSA-sig is used as the authentication method. The IKE_AUTH exchange usually includes the certificate containing the public key with the cryptographic relationship to the private key that is used to sign the data. There is a case where the certificate used for authentication is not sent within the exchange; however it is also possible to send an SHA-1 hash of the certificate and a URL indicating the receiving host can obtain this; this is known a HASH and URL or HTTP-URL-Lookup.

The AUTH payload consisting of the following:

AUTH = prf( prf(Shared Secret, “Key Pad for IKEv2”),<SignedOctets>)

It is worth noting the following problem. When a shared secret is used, if the initiator is tricked into connecting to a rogue VPN headend or if an MITM masquerades as the true responder, the AUTH payload is protected with the cryptographic material exchanged in the SA_INIT exchange. However, if an attacker has spoofed a response in which the initiator sends the authentication data, the attacker will be able to decrypt this and thus know all attributes used to generate the AUTH payload, apart from the shared secret. An attacker would be able to capture the AUTH payload and perform an offline brute-force attack, entering the correct (known) data for <SignedOctets> and the string “Key Pad for IKEv2,” while substituting values for the shared secret (for example, using a dictionary or brute force attack) until the value generated matched the value received. If the shared secret was a weak value using a dictionary word or a string with little entropy, it would be easily breakable with current computing power.

As IPsec can use Encapsulation Security Payload, or Authentication Header, or both protocols, there is a requirement to negotiate this before the IPsec Security Association is created. This negotiation occurs within the second IKE exchange, known as IKE_AUTH.

Figure 1-1 illustrates the IKEv2 Encrypted payload format, where the use of Encapsulation Security Payload can be seen. It should be noted that this differs from the Encapsulation Security Payload header as described in RFC4303; these is no SPI, although this is contained within the IKEv2 header itself.

The following illustrates a Proposal being negotiated by IKEv2 that will use Encapsulation Security Payload, which is Security Protocol ID 3 (with Authentication Header being ID 4).

Click here to view code image

Type Payload: Security Association (33)
                    Next payload: Traffic Selector – Initiator (44)
                    0... .... = Critical Bit: Not Critical
                    Payload length: 144
                    Type Payload: Proposal (2) # 1
                        Next payload: Proposal (2)
                        0... .... = Critical Bit: Not Critical
                        Payload length: 40
                        Proposal number: 1
                        Protocol ID: ESP (3)
                        SPI Size: 4
                        Proposal transforms: 3
                        SPI: cf1346e0

Each protocol is covered in the sections that follow.

Data origin authentication and connectionless integrity are joint services, which can be commonly referred to as integrity, which is provided by the use of the Integrity Check Value (ICV) field within the authentication header, which contains keyed Message Authentication Codes, also known as cryptographic keyed hash. The ICV is generated by taking the message as an input along with a secret key and producing a fixed length hash output. This hash will be the value of the ICV and is determined by the integrity algorithm used.

The Authentication Header uses IP protocol 51; therefore the protocol header (IPv4, IPv6, or IPv6 Extension) immediately before the Authentication Header header will contain the value 51 in its Protocol (IPv4) or Next Header (IPv6, Extension) field.

With the Authentication Header, the whole of the header (except for any mutable fields) is authenticated, along with all payload data.

Figure 1-2 illustrates both transport mode and tunnel mode when using Authentication Header with generic routing encapsulation (GRE). In all cases the outer IP header and subsequent data (including the Authentication Header) is authenticated. The modes of IPsec and GRE are described in detail in chapter 4, “IOS IPsec Implementation.”

As all fields in the packet headers are authenticated, if the outer IP address field changes, as would happen if using Network Address Translation (NAT) or Port Address Translation (PAT), then the receiving host would fail the authentication check when recalculating the ICV. For this reason Authentication Header cannot be used when NAT or PAT is used and is rarely used in customer deployments. As an alternative to using Authentication Header, Encapsulation Security Payload can be used with a null encryption (Encapsulation Security Payload-NULL). In this case only authentication and anti-replay services are provided by Encapsulation Security Payload, with no encryption service.

Figure 1-3 illustrates the Authentication Header format.

The use of the anti-replay window mitigates the situation in which a malicious observer captures traffic and then replays it. If these checks were not implemented, the replayed traffic would be correctly decrypted and reach the intended destination. This traffic could cause congestion on links and disruption on the receiver.

On most networks today, there will be diverse routes, devices with bundled interfaces (port-channels), queuing mechanisms, and devices that process packets by parallel means (multiple CPUs or cores servicing a packet queue). These have the potential to make packets that were sent sequentially be received out of order.

The IPsec sliding window specifies the number of packets that can be received and be processed by the receiver that arrive late. The window begins at the sequence number for the last packet that was received and correctly decrypted (for Encapsulation Security Payload) or authenticated (when using Authentication Header). This value is known as X. If the next packet’s sequence number is above the previous received sequence number or within X minus the sliding window value, then the packet will be processed.

As an example, if the sliding window was 100 packets and a packet with sequence number 522 was received, only packets with a sequence number 422 or higher would be processed.

Encapsulation Security Payload uses IP protocol 50, and the protocol header (IPv4, IPv6, or IPv6 Extension) immediately prior to the Encapsulation Security Payload header will contain the value 50 in the Protocol (IPv4) or Next Header (IPv6, Extension) field.

Figure 1-4 illustrates the Encapsulation Security Payload header format.

Unlike the Authentication Header, in which all fields in the packet headers are authenticated, Encapsulation Security Payload will not authenticate the outer IP header, which allows for any field to change the header without affecting the ICV, as would happen if Network Address Translation (NAT) or Port Address Translation (PAT) was in use. This results in Encapsulation Security Payload being compatible with NAT or PAT.

Encapsulation Security Payload version 3 allows for 64-bit sequence numbers, also known as extended sequence numbers (ESN) that increases the number of packets sent before a rekey will occur to 2⁶⁴–1 or 18,446,744,073,709,551,615 packets; this provides traffic to flow constantly through an IPsec protected 10Gbps interface for 100 years without a rekey.

The ability to use ESN is negotiated when the IPsec Security Association is negotiated.

The IPsec transport mode overhead compared to the original packet is just calculated as the crypto overhead (see chapter 15, “Calculating IPsec Overhead and Fragmentation”).

Figure 1-6 shows the encapsulation of a UDP over IPv4 packet into Encapsulation Security Payload transport mode with IPv4 transport. The UDP traffic endpoints are 10.x.x.x, with the packet being encrypted, and when the Encapsulation Security Payload header is inserted, it comes after the original IP header. The protocol field in the outer IP header is set to 50 (0x32) which denotes the next header is Encapsulation Security Payload.

This type of protection would be possible only with crypto maps, where traffic is protected when passing through an interface, and not with tunnel interfaces (which are covered in chapter 4, “IOS IPsec implementation”), because if a tunnel interface was configured for pure IPsec and transport mode was used, the IP header of the protected packet would be used after Encapsulation Security Payload protection. This is considered a misconfiguration, and at the time of this writing, Cisco IOS actually would autocorrect this to use tunnel mode instead of transport mode if a transform set of type mode transport is used without the additional required keyword.

The IPsec tunnel mode overhead compared to the original packet is:

20 bytes (more if IP options are present) for the outer IPv4 header, or

40 bytes (more if extension headers are present) for the outer IPv6 header, plus the crypto overhead (see chapter 15, “Calculating IPsec Overhead and Fragmentation”).

Figure 1-7 shows the encapsulation of a UDP over IPv4 packet into Encapsulation Security Payload tunnel mode with IPv4 transport. The UDP traffic endpoints are 10.x.x.x, and the IPsec tunnel endpoints are 172.16.x.x. The protocol field in the outer IP header is set to 50 (0x32), which denotes the next header is Encapsulation Security Payload.

The Internet Key Exchange was introduced within this chapter; although IKE is the protocol used to create and maintain IPsec Security Associations, IPsec itself is a separate entity.

The chapter discussed the databases used by IPsec and the ways these integrate together; the Security Association Database defines what is protected, and the Security Policy Database details how traffic is protected.

Encapsulation Security Payload version 3 introduces a number of security features, which include traffic flow confidentiality, “dummy” packets, and extended sequence numbers.

Tunnel mode allows for traffic to be protected within the IPsec Security Association, which performs limited traffic flow confidentiality by hiding the internal IP address, allowing for traffic to be tunneled within the IPsec Security Association. However, it requires an additional IP header. Transport mode requires that the protected traffic’s IP header is used to transport the packet on the wire, but it results in less overhead than Tunnel mode.

https://tools.ietf.org/html/rfc4301 Security Architecture for the Internet Protocol

https://tools.ietf.org/html/rfc4302 IP Authentication Header

https://tools.ietf.org/html/rfc4303 IP Encapsulating Security Payload

https://tools.ietf.org/html/rfc4304 Extended Sequence Number (ESN)