Transport mode

Applies To: Windows Server 2003, Windows Server 2003 R2, Windows Server 2003 with SP1, Windows Server 2003 with SP2

Transport mode

Transport mode is the default mode for IPSec, and it is used for end-to-end communications (for example, for communications between a client and a server). When transport mode is used, IPSec encrypts only the IP payload. Transport mode provides the protection of an IP payload through an AH or ESP header. Typical IP payloads are TCP segments (containing a TCP header and TCP segment data), a UDP message (containing a UDP header and UDP message data), or an ICMP message (containing an ICMP header and ICMP message data).

Authentication Header transport mode

Authentication Header (AH) provides authentication, integrity, and anti-replay protection for the entire packet (both the IP header and the data payload carried in the packet). It does not provide confidentiality, which means that it does not encrypt the data. The data is readable, but protected from modification. AH uses keyed hash algorithms to sign the packet for integrity. For more information, see Data integrity with hash functions.

For example, Alice on Computer A sends data to Bob on Computer B. The IP header, the AH header, and the data are protected with integrity. This means Bob can be certain it was really Alice who sent the data and that the data was unmodified.

Integrity and authentication are provided by the placement of the AH header between the IP header and the IP payload, as shown in the following illustration.

AH is identified in the IP header with an IP protocol ID of 51. AH can be used alone or combined with the Encapsulating Security Payload (ESP) protocol.

The AH header contains the following fields:

• Next Header

  Identifies the IP payload by using the IP protocol ID. For example, a value of 6 represents TCP.

• Length

  Indicates the length of the AH header.

• Security Parameters Index (SPI)

  Used in combination with the destination address and the security protocol (AH or ESP) to identify the correct security association for the communication. The receiver uses this value to determine with which security association the packet is identified.

• Sequence Number

  Provides anti-replay protection for the packet. The sequence number is a 32-bit, incrementally increasing number (starting from 1) that indicates the packet number sent over the security association for the communication. The sequence number cannot repeat for the life of the quick mode security association. The receiver checks this field to verify that a packet for a security association with this number has not already been received. If one has been received, the packet is rejected.

• Authentication Data

  Contains the integrity check value (ICV), also known as the message authentication code, which is used to verify both message authentication and integrity. The receiver calculates the ICV value and checks it against this value (which is calculated by the sender) to verify integrity. The ICV is calculated over the IP header, the AH header, and the IP payload.

Packet signature with the AH header

AH signs the entire packet for integrity, with the exception of some fields in the IP header which might change in transit (for example, the Time to Live and Type of Service fields). If another IPSec header is being used in addition to AH, the AH header is inserted before any other IPSec headers. The AH packet signature is shown in the following illustration.

Encapsulating Security Payload transport mode

Encapsulating Security Payload (ESP) provides confidentiality (in addition to authentication, integrity, and anti-replay protection) for the IP payload. ESP in transport mode does not sign the entire packet. Only the IP payload (not the IP header) is protected. ESP can be used alone or in combination with AH.

For example, Alice on Computer A sends data to Bob on Computer B. The IP payload is encrypted and signed for integrity. Upon receipt, after the integrity verification process is complete, the data payload in the packet is decrypted. Bob can be certain that it was Alice who sent the data, the data is unmodified, and no one else was able to read it.

ESP is identified in the IP header with the IP protocol ID of 50. As shown in the following illustration, the ESP header is placed before the IP payload, and an ESP trailer and ESP authentication trailer is placed after the IP payload.

The ESP header contains the following fields:

• Security Parameters Index

  Identifies the correct security association for the communication when used in combination with the destination address and the security protocol (AH or ESP). The receiver uses this value to determine the security association with which this packet should be identified.

• Sequence Number

  Provides anti-replay protection for the packet. The sequence number is a 32-bit, incrementally increasing number (starting from 1) that indicates the packet number sent over the quick mode security association for the communication. The sequence number cannot repeat for the life of the quick mode security association. The receiver checks this field to verify that a packet for a security association with this number has not already been received. If one has been received, the packet is rejected.

The ESP trailer contains the following fields:

• Padding

  Padding of 0 to 255 bytes is used to ensure that the encrypted payload with the padding bytes are on byte boundaries required by encryption algorithms.

• Padding Length

  Indicates the length of the Padding field in bytes. The receiver uses this field to remove padding bytes after the encrypted payload with the padding bytes has been decrypted.

• Next Header

  Identifies the type of data in the payload, such as TCP or UDP.

The ESP authentication trailer contains the following field:

• Authentication Data

  Contains the integrity check value (ICV), also known as the message authentication code, which is used to verify both message authentication and integrity. The receiver calculates the ICV value and checks it against this value (which is calculated by the sender) to verify integrity. The ICV is calculated over the ESP header, the payload data, and the ESP trailer.

Packet signature and encryption

As shown in the following illustration, ESP provides protection for IP payloads. The signed portion of the packet indicates where the packet has been signed for integrity and authentication. The encrypted portion of the packet indicates what information is protected with confidentiality.

The IP header is not signed and is not necessarily protected from modification. To provide data integrity and authentication for the IP header, use ESP and AH.