Internet Protocol Security (IPsec)

The work on the IP security suite protocol (IPsec) is carried out in the ipsec working group (www.ietf.org/html.charters/ipsec-charter.html) in the security area of the IETF. We strongly recommend Kaufman and Newman (1999) if you would like a detailed treatment of the subject and its business applications.

RFC 2401 defines the design goal of the IP security (IPsec) protocol as the means to provision interoperable, high-quality, cryptography-based security for both versions of IP protocol—IPv4 and IPv6. In a nutshell, the IPsec suite provides privacy (through encryption) and authentication services at the IP layer (which, among other things, means that application and transport security as well as data link layer security are outside of the scope of the IPsec suite). The services provided by the suite include access control, connectionless integrity, confidentiality, and protection against replays (that is, reuse of snooped messages).

To provide these services, first, two traffic security protocols are defined:

§  Authentication Header (AH) (RFC 2402). Provides connectionless integrity, data origin authentication, and an (optional) antireplay service.

§  Encapsulating Security Payload (ESP) (RFC 2406). Provides one of the two sets of services:

1.  Confidentiality and limited traffic flow confidentiality.

2.  Integrity, data origin authentication, and an antireplay service.

Each of these protocols can be used in either transport mode (where transport protocols are protected) or tunnel mode (where the IP itself is protected). Tunnel mode is particularly important to VPN services in connection with IPIP (RFC 2003).

Second, algorithms for encryption and authentication as well as automatic key management schemes (such as RFC 2408 and RFC 2409) are defined, each in a separate RFC. The informational RFC 2411 provides guidelines for specifying new encryption and authentication algorithms and explains the interrelationship of all the protocols and algorithms in the IPsec suite.

The interrelationship of the elements of the IPsec architecture is demonstrated in Figure 1. The ESP and AH protocol documents cover the packet format and general issues of these protocols. They also define the default and mandatory values (as, for example, specified in RFC 2407) that are fed into the domain of interpretation (DOI) document, which deals with assigned values. The ESP protocols can use various encryption algorithms (for example, those specified by RFC 2405 and RFC 2451) as well as authentication algorithms (for example, those specified by RFC 2403 and RFC 2404). The authentication algorithms are also used by the AH protocol.

Figure 1: IPsec architecture.

Source: RFC 2411.

Both the encryption and authentication algorithms may also specify values for the DOI (for example, the values that identify these algorithms). Finally, the key management documents specify the IETF standards-track key management schemes, which also define the values for DOI. The IPsec DOI also supports negotiation of IP compression (as specified in RFC 2393); this function may be necessary because the encryption of IP payload prevents compression at lower layers. For example, a message containing a string of 50 octets filled with 0s could have been successfully compressed to something much smaller than 50 octets had it not been for the encryption, which—by design—not only changed 0s to other values but also eliminated the pattern.

Overall, IPsec provides security services at the IP layer by selecting (1) appropriate security protocols, (2) algorithms to be used with these protocols, and (3) appropriate cryptographic keys. The major function of the Internet Key Exchange (IKE) protocol (RFC 2409) is to establish and manage security associations (SAs), a function supported by both AH and ESP.

The concept of SAs is fundamental to IPsec. RFC 2401 defines an SA as a simplex connection that affords security services to the traffic carried by it. These services can be afforded by either AH or ESP, but in the case where both are to be employed, two SAs need to be established. An SA is uniquely identified by the following triplet:

1.  Security parameter index (SPI).

2.  IP destination address. Although this can in principle also be a broadcast or multicast address, only the unicast address is presently supported by the IPsec management mechanism.

3.  Security protocol identifier (that is, AH or ESP).

RFC 2401 defines two types of SAs—transport mode SA and tunnel mode SA—to support the AH and ESP modes of operation, respectively. As Figure 2 demonstrates, the transport mode SA is an association between two hosts. The double arrows are used only to indicate the nodes between which the SAs are established, not to imply by any means that that the SAs are duplex! The tunnel mode SA, however, may be established between:

§  Two security gateways. A security gateway is defined in RFC 2401 as an intermediate system that acts as the communications interface between two networks.

§  A host and a security gateway.

§  Two hosts.

Figure 2: Two types of security associations (SAs).

Any SA that supports tunneling (that is, an SA between a host and a security gateway or two security gateways) must be a tunnel mode SA, but an SA between two hosts may be either a transport mode SA or tunnel mode SA. The modes define the order and selection of appropriate headers. RFC 2401 provides several interesting examples of how SAs can be combined to satisfy complex, nesting security policies, and introduces into the model the notions of the security policy database (which specifies the policies that determine the disposition of IP traffic) and the security association database (which specifies the parameters of each active SA). Finally, RFC 2401 studies four cases of security associations, of which the fourth—depicted in Figure 3—is specifically relevant to the topic of dial-in access to an enterprise network. 

Figure 3: Remote host reaching an enterprise server.

Source: RFC 2401, Case 4.

In Figure 3, a remote host (dialing into an access concentrator) uses the Internet to reach an enterprise’s firewall (a security gateway) and then a particular host (a server). Only tunnel mode is required for an SA between the remote host and the firewall, while an SA between the remote host and the server can be either a transport or tunnel mode SA.

We have mentioned how IPsec standards relate to access and, particularly, VPNs, but so far we have not said anything about IPsec’s relation to IP telephony. The general feeling in the IP telephony community^[2] is that AH and ESP can be used in a standalone IP telephone, but the complexities of key exchange through IKE and the Internet Security Association and Key Management Protocol (ISAKMP) are too much to be handled by a client in a small standalone telephone.

Kaufman and Newman (1999) report that “early generations of IPsec products have proven to be suboptimal for many VoIP installations” and recommend deploying IPsec and voice over Internet Protocol (VoIP) separately and “experiment[ing] with integration only when you have debugged both installations.” In addition to technical reasons, the same monograph brings up an important regulatory consideration: In some circumstances, there are regulations that restrict voice encryption.