IPsec And…? - Using IPsec With Other Encryption Methods

====================== DISCLAIMER ====================
This posting is provided "AS IS" with no warranties, and confers no rights.
====================================================

The question is “What combinations of security protocols can be used with IPsec and which ones cannot?” It’s a fair questions and reasonable given that your IT environments can be rather heterogeneous and use PKI, PGP, IPsec, SSL, and WKWE (Who-Knows-What-Else).

IPsec + [ SSL || PGP ||?]= Yes
As mentioned in an earlier article, IPsec and SSL are friends and you can use them in combination with no difficulties. This is true because SSL encrypts the data that will go into the IP Datagram that IPsec will, in turn, encrypt. So you get double encryption.

IPsec + PKI= Yes (and No)
If you are using PKI (asymmetric key cryptography) at the Application Layer, then Yes, for the same reasons as SSL and PGP are compatible.

However, the answer to the question “Can I use PKI keys in IPsec to encrypt the ESP Payloads?” is a definite No. The Windows implementation of IPsec uses a set of predefined algorithms for data integrity (MD5 or SHA1) and encryption (DES or 3DES) and PKI is not one of them. The reason for this is that PKI (asymmetric) keys are not suitable for bulk data encryption, due to performance and other reasons, however they are can be used to authenticate the end points (hosts).