Ipsec site to site tunnel between ISA and TMG server Fails with "Peer failed with Windows error 13802(ERROR_IPSEC_IKE_ATTRIB_FAIL)" in IKEext.etl logs
Issue: IPSEC site to site tunnel between ISA and TMG server Fails.
Scenario: Admin is trying to establish site to site IPsec tunnel between ISA server 2006 on one end and TMG 2010 on other end but tunnel does not get established. After configuring tunnel on both ends admin pings from TMG 2010 machine to domain controller on remote end and its gets request time out
Site1
Domain: mylabisa.local
TMG 2010 : External NIC IP address : 10.171.82.55
Internal NIC IP address : 192.168.0.124
Machine name : TMG-node1
site to site ipsec tunnel between two sites
Access rule : Allows all protocols/internal+localhost+site-to-site network(to and from)
Network rule: Route relation between site-to-site and internal network
Site 2
Domain: Domain2.local
ISA server 2006 : External NIC IP address : 10.171.82.104
Internal NIC IP address : 172.16.0.1
Machine name : ISAdomain2
site to site ipsec tunnel between two sites
Access rule : Allows all protocols/internal+localhost+site-to-site network(to and from)
Network rule: Route relation between site-to-site and internal network
Data collection
For the proof of concept I collected the data using TMG data packager using vpn template(refer) while doing a ping to correlate IKEext.etl logs to what could be causing the issue.
Data analysis and Solution
Formatted IKEext.etl logs collected by TMG data packager and found following
Name: ISA VPN S2S tunnel to network site-2-site
Description: (null)
Flags: 0x00000000
Provider: <unspecified>
Provider data:
Type: IKE Main Mode
Soft expiry: 28800
InitiatorImpersonationType: None
Auth methods: 1
-- 0 --
Type: Preshared Key
Key:
00000000 31 32 33 123
Flags: 0x00000000
Proposals: 1
-- 0 --
Cipher algorithm:
Type: AES-256
Key length: 0
Rounds: 0
Integrity algorithm:
Type: SHA_256
Max lifetime (sec): 7200
DH group: DH-2
QM limit: 0
Flags: 0x00000000
MaxDynamicFilters: 5
RetransmitDuration: 0
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|Inserted ICookie into the high priority thread pool list
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|Construct IKEHeader
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|Initializing Kerberos SSPI
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|ACH: 0(ERROR_SUCCESS)
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|My SPN: tmg-node1$@MYLABISA.LOCAL
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|Construct SA
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|AUTHIP keying module is not enabled for traffic
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|IKE not sending co-existence Vendor ID
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|Construct VENDOR type MS NT5 ISAKMPOAKLEY
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|Construct VENDOR type RFC 3947
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|Construct VENDOR type draft-ietf-ipsec-nat-t-ike-02
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|Construct VENDOR type FRAGMENTATION
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|Construct VENDOR type MS-Negotiation Discovery Capable
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|Construct VENDOR type Vid-Initial-Contact
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|Construct VENDOR type IKE CGA version 1
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|Sending Packet
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|iCookie ae1067e95973fb37 rCookie 0000000000000000
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|Exchange type: IKE Main Mode Length 232 NextPayload SA Flags 0x0 Messid 0x00000000
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|Local Address: 10.171.82.55.500 Protocol 0
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|Peer Address: 10.171.82.104.500 Protocol 0
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|IF-Luid: 1688849994481664
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|Global IF index epoch (89), cache epoch (0). Querying IF index.
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|IF-Index: 13
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|Created new TimerContext 0000000001FE0880, type 0
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|Processing acquire with ipsec context 158, keyMod 1
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|QM localAddr: 10.171.82.55.0 Protocol 0
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|QM peerAddr : 172.16.0.0.0 Mask 255.255.0.0 Protocol 0
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|IF-Luid: 1688849994481664
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|Profile ID: 1
[0]0344.0B48::05/14/2011-20:27:00.519 [ikeext] 0|10.171.82.104|Acquire flags 1
[0]0344.0B48::05/14/2011-20:27:00.520 [ikeext] 0|10.171.82.104|FwpmFilterEnum returned no matching filters
[0]0344.0B48::05/14/2011-20:27:00.520 [user] |10.171.82.104|IkeMatchFwpmFilter failed with Windows error 13825(ERROR_IPSEC_IKE_NO_POLICY)
[0]0344.0B48::05/14/2011-20:27:00.520 [user] |10.171.82.104|IkeMatchFwpmFilter failed with HRESULT 0x80073601(ERROR_IPSEC_IKE_NO_POLICY)
[0]0344.0B48::05/14/2011-20:27:00.520 [user] |10.171.82.104|IkeFindQMPolicy failed with HRESULT 0x80073601(ERROR_IPSEC_IKE_NO_POLICY)
[0]0344.0B48::05/14/2011-20:27:00.520 [ikeext] 0|10.171.82.104|Completing Acquire for ipsec context 158
[0]0344.0B48::05/14/2011-20:27:00.520 [ikeext] 0|10.171.82.104|IkeFreeAcquireContext: Freeing acquire 0000000001FE0D70
[0]0344.0B48::05/14/2011-20:27:00.520 [user] |NULL|IkeProcessAcquireDispatch failed with HRESULT 0x80073601(ERROR_IPSEC_IKE_NO_POLICY)
[0]0344.0F20::05/14/2011-20:27:00.523 [ikeext] 0|10.171.82.104|Original arrival IF index 13
[0]0344.0F20::05/14/2011-20:27:00.523 [ikeext] 0|10.171.82.104|Queuing work to high priority thread pool
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 0|10.171.82.104|
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 0|10.171.82.104|Received packet
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 0|10.171.82.104|Local Address: 10.171.82.55.500 Protocol 0
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 0|10.171.82.104|Peer Address: 10.171.82.104.500 Protocol 0
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 0|10.171.82.104|IF-Luid: 1688849994481664
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 0|10.171.82.104|Profile ID: 1
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 0|10.171.82.104|iCookie ae1067e95973fb37 rCookie 5cecc0f08440ab99
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 0|10.171.82.104|Exchange type: IKE Informational Mode Length 68 NextPayload NOTIFY Flags 0x0 Messid 0x702d38a5
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 636|10.171.82.104|mmSa: 0x00000000017EB960
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 636|10.171.82.104|Process Payload OAK NOTIFY/DELETE, SA 00000000017EB960
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 636|10.171.82.104|Processing Cryptoless notify, SA 00000000017EB960
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 636|10.171.82.104|Oak NotifyMessage 14
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 636|10.171.82.104|Peer sent NO_PROPOSAL_CHOSEN notify
[0]0344.0C4C::05/14/2011-20:27:00.523 [user] |10.171.82.104|Peer failed with Windows error 13802(ERROR_IPSEC_IKE_ATTRIB_FAIL)
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 636|10.171.82.104|ProcessNotifyData: mmSa 00000000017EB960 cookie e96710ae state 1 messId 0
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 636|10.171.82.104|Deleted ICookie from the high priority thread pool list
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 636|10.171.82.104|IKE diagnostic event:
Event Header:
Timestamp: 1601-01-01T00:00:00.000Z
Flags: 0x00000106
Local address field set
Remote address field set
IP version field set
IP version: IPv4
IP protocol: 0
Local address: 10.171.82.55
Remote address: 10.171.82.104
Local Port: 0
Remote Port: 0
Application ID:
User SID: <invalid>
Failure type: IKE/Authip Main Mode Failure
Type specific info:
Failure error code:0x000035ea
IKE security attributes are unacceptable
Failure point: Remote
Flags: 0x00000000
Keying module type: Ike
MM State: First roundtrip packet sent
MM SA role: Initiator
MM auth method: Unknown
Cert hash:
0000000000000000000000000000000000000000
MM ID: 0x000000000000027c
MM Filter ID: 0x0000000000014e89
Local Principal Name: tmg-node1$@MYLABISA.LOCAL
Remote Principal Name:
Local Principal Group SIDs:
Remote Principal Group SIDs:
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 636|10.171.82.104|Cleaning up mmSa: 00000000017EB960. Error 13802(ERROR_IPSEC_IKE_ATTRIB_FAIL)
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 636|10.171.82.104|Destroying TimerContext 0000000001FE0880, type 0
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 636|10.171.82.104|TimerContext 0000000001FE0880, Old ref 2
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 636|10.171.82.104|TimerContext 0000000001FE0880, Old ref 1
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 636|10.171.82.104|Freeing TimerContext 0000000001FE0880
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 636|10.171.82.104|Inactivating MM: 00000000017EB960
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 636|10.171.82.104|Moving mmSa 00000000017EB960 to zombie list
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 636|10.171.82.104|Deleting MM from lists: 00000000017EB960
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 636|10.171.82.104|Completing Acquire for ipsec context 158
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 636|10.171.82.104|IkeFreeAcquireContext: Freeing acquire 0000000001FE0B80
[0]0344.0C4C::05/14/2011-20:27:00.523 [ikeext] 636|10.171.82.104|WFP free sspi 00000000017F4020
[0]0344.0F20::05/14/2011-20:27:14.832 [ikeext] 0|10.171.82.104|Original arrival IF index 13
[0]0344.0A20::05/14/2011-20:27:14.833 [ikeext] 0|10.171.82.104|
[0]0344.0A20::05/14/2011-20:27:14.833 [ikeext] 0|10.171.82.104|Received packet
[0]0344.0A20::05/14/2011-20:27:14.833 [ikeext] 0|10.171.82.104|Local Address: 10.171.82.55.500 Protocol 0
[0]0344.0A20::05/14/2011-20:27:14.833 [ikeext] 0|10.171.82.104|Peer Address: 10.171.82.104.500 Protocol 0
[0]0344.0A20::05/14/2011-20:27:14.833 [ikeext] 0|10.171.82.104|IF-Luid: 1688849994481664
[0]0344.0A20::05/14/2011-20:27:14.833 [ikeext] 0|10.171.82.104|Profile ID: 1
[0]0344.0A20::05/14/2011-20:27:14.833 [ikeext] 0|10.171.82.104|iCookie 1062fe261a891a22 rCookie 121feda274e2ad89
[0]0344.0A20::05/14/2011-20:27:14.833 [ikeext] 0|10.171.82.104|Exchange type: IKE Informational Mode Length 56 NextPayload DELETE Flags 0x0 Messid 0xeb02c45f
[0]0344.0A20::05/14/2011-20:27:14.833 [ikeext] 0|10.171.82.104|Dropping non-initial MM packet with no state to process
[0]0344.0F20::05/14/2011-20:27:59.827 [ikeext] 0|10.171.82.104|Original arrival IF index 13
[0]0344.0A20::05/14/2011-20:27:59.827 [ikeext] 0|10.171.82.104|
[0]0344.0A20::05/14/2011-20:27:59.827 [ikeext] 0|10.171.82.104|Received packet
[0]0344.0A20::05/14/2011-20:27:59.827 [ikeext] 0|10.171.82.104|Local Address: 10.171.82.55.500 Protocol 0
[0]0344.0A20::05/14/2011-20:27:59.827 [ikeext] 0|10.171.82.104|Peer Address: 10.171.82.104.500 Protocol 0
[0]0344.0A20::05/14/2011-20:27:59.827 [ikeext] 0|10.171.82.104|IF-Luid: 1688849994481664
[0]0344.0A20::05/14/2011-20:27:59.827 [ikeext] 0|10.171.82.104|Profile ID: 1
[0]0344.0A20::05/14/2011-20:27:59.827 [ikeext] 0|10.171.82.104|iCookie ae1067e95973fb37 rCookie 5cecc0f08440ab99
[0]0344.0A20::05/14/2011-20:27:59.827 [ikeext] 0|10.171.82.104|Exchange type: IKE Informational Mode Length 56 NextPayload DELETE Flags 0x0 Messid 0xba1b4a76
[0]0344.0A20::05/14/2011-20:27:59.827 [ikeext] 0|10.171.82.104|Dropping non-initial MM packet with no state to process
[0]0344.0134::05/14/2011-20:28:15.455 [ikeext] 0|NULL|IkeRegConfigChangeNotifyCallback invoked
[0]0344.0134::05/14/2011-20:28:15.455 [ikeext] 0|NULL|Stopping IKE tracing
13825(ERROR_IPSEC_IKE_NO_POLICY) error usualy means that policies are not configured properly so checked the configuration on the TMG server and found phase 1(main mode) has following settings
compared it with the phase 1(main mode) IPsec settings on the ISA server's tunnel and found following
which clearly shows the difference in settings i.e. TMG is configured with AES256 encryption and ISA is with 3DES similarly TMG is configured with SHA256 integrity and ISA is configured with SHA1.
Since we cannot configure ISA with higher values used by TMG for encryption and integrity we changed the values on TMG to match with that of ISA server. After making the changes we got our tunnel working.
Comments
- Anonymous
May 16, 2011
Good Job :-)