A case study on System.InvalidOperationException due to SQLConnection leak
I ran into this exception below while I was load testing TFS last week:
System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool.
This may have occurred because all pooled connections were in use and max pool size was reached.
Basically this means that the connection pool is full and that all connections are in use.
1. Verified SQL connections are indeed leaked
I used Perf counter ".Net Data Provider for SqlServer"\NumberOfRelaimedConnections to verify we indeed have a leak. This perf counter keeps going up during my load run, which means that a lot of SQL connections are not closed properly by our code.
2. Verify SQL connection pool is maxed out
0:079> !dumpheap -stat -type System.Data.ProviderBase.DbConnectionPool
Statistics:
MT Count TotalSize Class Name
000007fedf56fe00 1 24 System.Collections.Generic.ObjectEqualityComparer`1[[System.Data.ProviderBase.DbConnectionPoolIdentity, System.Data]]
...
000007fedf56ffd8 2 192 System.Collections.Generic.Dictionary`2+Entry[[System.Data.Common.DbConnectionPoolKey, System.Data],[System.Data.ProviderBase.DbConnectionPoolGroup, System.Data]][]
000007fedf56fe88 4 192 System.Collections.Concurrent.ConcurrentDictionary`2+Node[[System.Data.ProviderBase.DbConnectionPoolIdentity, System.Data],[System.Data.ProviderBase.DbConnectionPool, System.Data]]
000007fedf386ec0 4 224 System.Collections.Concurrent.ConcurrentDictionary`2[[System.Data.ProviderBase.DbConnectionPoolIdentity, System.Data],[System.Data.ProviderBase.DbConnectionPool, System.Data]]
000007fedf56ed90 14 336 System.Data.ProviderBase.DbConnectionPoolCounters+Counter
000007fedf56bfc8 2 336 System.Data.ProviderBase.DbConnectionPool
000007fee1cbeb28 10 2944 System.Object[]
Total 63 objects
0:079> !dumpheap -mt 000007fedf56bfc8
Address MT Size
000000047fd3a260 000007fedf56bfc8 168
000000047fd5f468 000007fedf56bfc8 168
Statistics:
MT Count TotalSize Class Name
000007fedf56bfc8 2 336 System.Data.ProviderBase.DbConnectionPool
Total 2 objects
0:079> !do 000000047fd3a260
Name: System.Data.ProviderBase.DbConnectionPool
MethodTable: 000007fedf56bfc8
EEClass: 000007fedf3cfd48
Size: 168(0xa8) bytes
File: D:\Windows\Microsoft.Net\assembly\GAC_64\System.Data\v4.0_4.0.0.0__b77a5c561934e089\System.Data.dll
Fields:
MT Field Offset Type VT Attr Value Name
000007fee1cf6408 4000e70 80 System.Int32 1 instance 190000 _cleanupWait
000007fedf56ef90 4000e71 8 ...ctionPoolIdentity 0 instance 000000047fd3a240 _identity
000007fedf56a3c8 4000e72 10 ...ConnectionFactory 0 instance 000000047fd31438 _connectionFactory
000007fedf56df40 4000e73 18 ...nnectionPoolGroup 0 instance 000000047fd36d10 _connectionPoolGroup
000007fedf56cec0 4000e74 20 ...nPoolGroupOptions 0 instance 000000047fd36ce8 _connectionPoolGroupOptions
000007fedf57e7a8 4000e75 28 ...nPoolProviderInfo 0 instance 0000000000000000 _connectionPoolProviderInfo
000007fedf577368 4000e76 84 System.Int32 1 instance 1 _state
000007fedf387188 4000e77 30 ...al, System.Data]] 0 instance 000000047fd3a3e8 _stackOld
000007fedf387188 4000e78 38 ...al, System.Data]] 0 instance 000000047fd3a408 _stackNew
000007fedf387240 4000e79 40 ...on, System.Data]] 0 instance 000000047fd3a428 _pendingOpens
000007fee1cf6408 4000e7a 88 System.Int32 1 instance 0 _pendingOpensWaiting
000007fee1ce3b90 4000e7b 48 ...ding.WaitCallback 0 instance 000000047fd3add0 _poolCreateRequest
000007fee1cf6408 4000e7c 8c System.Int32 1 instance 0 _waitCount
000007fedf568ff0 4000e7d 50 ...l+PoolWaitHandles 0 instance 000000047fd3a708 _waitHandles
000007fee1cf3888 4000e7e 58 System.Exception 0 instance 0000000000000000 _resError
000007fee1cf4f30 4000e7f 9c System.Boolean 1 instance 0 _errorOccurred
000007fee1cf6408 4000e80 90 System.Int32 1 instance 5000 _errorWait
000007fee1cf1fe8 4000e81 60 ...m.Threading.Timer 0 instance 0000000000000000 _errorTimer
000007fee1cf1fe8 4000e82 68 ...m.Threading.Timer 0 instance 000000047fd3aef0 _cleanupTimer
000007fedf569850 4000e83 70 ...tedConnectionPool 0 instance 000000047fd3abb8 _transactedConnectionPool
000007fedf387308 4000e84 78 ...al, System.Data]] 0 instance 000000047fd3a850 _objectList
000007fee1cf6408 4000e85 94 System.Int32 1 instance 7 _totalObjects
000007fee1cf6408 4000e87 98 System.Int32 1 instance 1 _objectID
000007fee1ce4810 4000e6f b88 System.Random 0 shared static _random
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:000000047fd3a5f0 <<
000007fee1cf6408 4000e86 9e0 System.Int32 1 shared static _objectTypeCount
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:4 <<
Ok, this Connection pool is not maxed out. Check next one.
0:079> !do 00000002bfe10368
Free Object
Size: 638840(0x9bf78) bytes
0:079> !do 000000047fd5f468
Name: System.Data.ProviderBase.DbConnectionPool
MethodTable: 000007fedf56bfc8
EEClass: 000007fedf3cfd48
Size: 168(0xa8) bytes
File: D:\Windows\Microsoft.Net\assembly\GAC_64\System.Data\v4.0_4.0.0.0__b77a5c561934e089\System.Data.dll
Fields:
MT Field Offset Type VT Attr Value Name
000007fee1cf6408 4000e70 80 System.Int32 1 instance 170000 _cleanupWait
000007fedf56ef90 4000e71 8 ...ctionPoolIdentity 0 instance 000000047fd3a240 _identity
000007fedf56a3c8 4000e72 10 ...ConnectionFactory 0 instance 000000047fd31438 _connectionFactory
000007fedf56df40 4000e73 18 ...nnectionPoolGroup 0 instance 000000047fd5f090 _connectionPoolGroup
000007fedf56cec0 4000e74 20 ...nPoolGroupOptions 0 instance 000000047fd5f068 _connectionPoolGroupOptions
000007fedf57e7a8 4000e75 28 ...nPoolProviderInfo 0 instance 0000000000000000 _connectionPoolProviderInfo
000007fedf577368 4000e76 84 System.Int32 1 instance 1 _state
000007fedf387188 4000e77 30 ...al, System.Data]] 0 instance 000000047fd5f510 _stackOld
000007fedf387188 4000e78 38 ...al, System.Data]] 0 instance 000000047fd5f530 _stackNew
000007fedf387240 4000e79 40 ...on, System.Data]] 0 instance 000000047fd5f550 _pendingOpens
000007fee1cf6408 4000e7a 88 System.Int32 1 instance 0 _pendingOpensWaiting
000007fee1ce3b90 4000e7b 48 ...ding.WaitCallback 0 instance 000000047fd5fc40 _poolCreateRequest
000007fee1cf6408 4000e7c 8c System.Int32 1 instance 20 _waitCount
000007fedf568ff0 4000e7d 50 ...l+PoolWaitHandles 0 instance 000000047fd5f718 _waitHandles
000007fee1cf3888 4000e7e 58 System.Exception 0 instance 0000000000000000 _resError
000007fee1cf4f30 4000e7f 9c System.Boolean 1 instance 0 _errorOccurred
000007fee1cf6408 4000e80 90 System.Int32 1 instance 5000 _errorWait
000007fee1cf1fe8 4000e81 60 ...m.Threading.Timer 0 instance 0000000000000000 _errorTimer
000007fee1cf1fe8 4000e82 68 ...m.Threading.Timer 0 instance 000000047fd5fcc0 _cleanupTimer
000007fedf569850 4000e83 70 ...tedConnectionPool 0 instance 000000047fd5fbc8 _transactedConnectionPool
000007fedf387308 4000e84 78 ...al, System.Data]] 0 instance 000000047fd5f860 _objectList
000007fee1cf6408 4000e85 94 System.Int32 1 instance 100 _totalObjects
000007fee1cf6408 4000e87 98 System.Int32 1 instance 3 _objectID
000007fee1ce4810 4000e6f b88 System.Random 0 shared static _random
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:000000047fd3a5f0 <<
000007fee1cf6408 4000e86 9e0 System.Int32 1 shared static _objectTypeCount
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:4 <<
This connection pool appears to be maxed out (default pool size is 100. Check _connectionPoolGroupOptions to see if default pool size is used).
0:079> !do 000000047fd5f068
Name: System.Data.ProviderBase.DbConnectionPoolGroupOptions
MethodTable: 000007fedf56cec0
EEClass: 000007fedf3d0438
Size: 40(0x28) bytes
File: D:\Windows\Microsoft.Net\assembly\GAC_64\System.Data\v4.0_4.0.0.0__b77a5c561934e089\System.Data.dll
Fields:
MT Field Offset Type VT Attr Value Name
000007fee1cf4f30 4000ed4 14 System.Boolean 1 instance 1 _poolByIdentity
000007fee1cf6408 4000ed5 8 System.Int32 1 instance 0 _minPoolSize
000007fee1cf6408 4000ed6 c System.Int32 1 instance 100 _maxPoolSize
000007fee1cf6408 4000ed7 10 System.Int32 1 instance 15000 _creationTimeout
000007fee1cf0658 4000ed8 18 System.TimeSpan 1 instance 000000047fd5f080 _loadBalanceTimeout
000007fee1cf4f30 4000ed9 15 System.Boolean 1 instance 1 _hasTransactionAffinity
000007fee1cf4f30 4000eda 16 System.Boolean 1 instance 0 _useLoadBalancing
Yep, one of the SQL connection pool is indeed maxed out.
3. Figure out which SQL connections are leaked and who is leaking them
Leaked SQL Connections are not GC rooted and their _closedCount field is set to 0.
0:079> !dumpheap -stat -type System.Data.SqlClient.SqlConnection
Statistics:
MT Count TotalSize Class Name
000007fedf56cac8 1 64 System.Data.SqlClient.SqlConnectionFactory
000007fedf3a5ce0 1 80 System.Collections.Generic.Dictionary`2[[System.String, mscorlib],[System.Data.SqlClient.SqlConnectionStringBuilder+Keywords, System.Data]]
000007fedf56ccc0 2 112 System.Data.SqlClient.SqlConnectionPoolGroupProviderInfo
000007fedf56ae10 2 400 System.Data.SqlClient.SqlConnectionString
000007fedfb11a88 1 1440 System.Collections.Generic.Dictionary`2+Entry[[System.String, mscorlib],[System.Data.SqlClient.SqlConnectionStringBuilder+Keywords, System.Data]][]
000007fedf56a818 107 4280 System.Data.SqlClient.SqlConnectionTimeoutErrorInternal
000007fee1cbeb28 107 11128 System.Object[]
000007fedf56daf0 749 17976 System.Data.SqlClient.SqlConnectionTimeoutPhaseDuration
000007fedf56a620 559 22360 System.Data.SqlClient.SqlConnectionPoolKey
000007fedf566170 588 75264 System.Data.SqlClient.SqlConnection
000007fedf577a00 557 102488 System.Data.SqlClient.SqlConnectionStringBuilder
Total 2674 objects
Dump out all Sql Connections so we can examine their _closedCount field: (We're looking for the ones with _ClosedCount=0)
0:079> .foreach (conn {!DumpHeap -short -MT 000007fedf566170}) {.echo foo; !do conn}
0000000100030770
Name: System.Data.SqlClient.SqlConnection
MethodTable: 000007fedf566170
EEClass: 000007fedf3a8110
Size: 128(0x80) bytes
File: D:\Windows\Microsoft.Net\assembly\GAC_64\System.Data\v4.0_4.0.0.0__b77a5c561934e089\System.Data.dll
Fields:
MT Field Offset Type VT Attr Value Name
000007fee1cf3c38 4000198 8 System.Object 0 instance 0000000000000000 __identity
000007fee0d6f2e8 40002d2 10 ...ponentModel.ISite 0 instance 0000000000000000 site
000007fee0d7dbe0 40002d3 18 ....EventHandlerList 0 instance 0000000100032178 events
000007fee1cf3c38 40002d1 1b8 System.Object 0 shared static EventDisposed
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:000000047fd55f50 <<
000007fedf581510 40002c0 20 ...hangeEventHandler 0 instance 0000000000000000 _stateChangeEventHandler
000007fedf57e4c0 400107c 28 ...t.SqlDebugContext 0 instance 0000000000000000 _sdc
000007fee1cf4f30 400107d 70 System.Boolean 1 instance 0 _AsyncCommandInProgress
000007fedf5863e0 400107e 30 ...ent.SqlStatistics 0 instance 0000000000000000 _statistics
000007fee1cf4f30 400107f 71 System.Boolean 1 instance 0 _collectstats
000007fee1cf4f30 4001080 72 System.Boolean 1 instance 0 _fireInfoMessageEventOnUserErrors
000007fedfb0de48 4001081 38 ....Task, mscorlib]] 0 instance 0000000000000000 _currentCompletion
000007fedf5698b0 4001082 40 ...ent.SqlCredential 0 instance 0000000000000000 _credential
000007fee1cf3658 4001083 48 System.String 0 instance 00000001000306a8 _connectionString
000007fedf56c0e0 4001086 50 ...ConnectionOptions 0 instance 0000000000000000 _userConnectionOptions
000007fedf56df40 4001087 58 ...nnectionPoolGroup 0 instance 0000000000000000 _poolGroup
000007fedf56d730 4001088 60 ...onnectionInternal 0 instance 000000047fd43d80 _innerConnection
000007fee1cf6408 4001089 68 System.Int32 1 instance 1 _closeCount
000007fee1cf6408 400108b 6c System.Int32 1 instance 322025 ObjectID
000007fee1cf3c38 400107b ea0 System.Object 0 shared static EventInfoMessage
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:000000047fd31420 <<
000007fedf56a3c8 4001084 ea8 ...ConnectionFactory 0 shared static _connectionFactory
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:000000047fd31438 <<
000007fee1ce8290 4001085 eb0 ...eAccessPermission 0 shared static ExecutePermission
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:000000047fd352f8 <<
000007fee1cf6408 400108a a08 System.Int32 1 shared static _objectTypeCount
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:322560 <<
...
00000004c0280050
Name: System.Data.SqlClient.SqlConnection
MethodTable: 000007fedf566170
EEClass: 000007fedf3a8110
Size: 128(0x80) bytes
File: D:\Windows\Microsoft.Net\assembly\GAC_64\System.Data\v4.0_4.0.0.0__b77a5c561934e089\System.Data.dll
Fields:
MT Field Offset Type VT Attr Value Name
000007fee1cf3c38 4000198 8 System.Object 0 instance 0000000000000000 __identity
000007fee0d6f2e8 40002d2 10 ...ponentModel.ISite 0 instance 0000000000000000 site
000007fee0d7dbe0 40002d3 18 ....EventHandlerList 0 instance 00000004c02819b8 events
000007fee1cf3c38 40002d1 1b8 System.Object 0 shared static EventDisposed
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:000000047fd55f50 <<
000007fedf581510 40002c0 20 ...hangeEventHandler 0 instance 0000000000000000 _stateChangeEventHandler
000007fedf57e4c0 400107c 28 ...t.SqlDebugContext 0 instance 0000000000000000 _sdc
000007fee1cf4f30 400107d 70 System.Boolean 1 instance 0 _AsyncCommandInProgress
000007fedf5863e0 400107e 30 ...ent.SqlStatistics 0 instance 0000000000000000 _statistics
000007fee1cf4f30 400107f 71 System.Boolean 1 instance 0 _collectstats
000007fee1cf4f30 4001080 72 System.Boolean 1 instance 0 _fireInfoMessageEventOnUserErrors
000007fedfb0de48 4001081 38 ....Task, mscorlib]] 0 instance 0000000000000000 _currentCompletion
000007fedf5698b0 4001082 40 ...ent.SqlCredential 0 instance 0000000000000000 _credential
000007fee1cf3658 4001083 48 System.String 0 instance 00000004c027ff88 _connectionString
000007fedf56c0e0 4001086 50 ...ConnectionOptions 0 instance 000000047fd5ed68 _userConnectionOptions
000007fedf56df40 4001087 58 ...nnectionPoolGroup 0 instance 000000047fd5f090 _poolGroup
000007fedf56d730 4001088 60 ...onnectionInternal 0 instance 00000003c00425d0 _innerConnection
000007fee1cf6408 4001089 68 System.Int32 1 instance 0 _closeCount
000007fee1cf6408 400108b 6c System.Int32 1 instance 322129 ObjectID
000007fee1cf3c38 400107b ea0 System.Object 0 shared static EventInfoMessage
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:000000047fd31420 <<
000007fedf56a3c8 4001084 ea8 ...ConnectionFactory 0 shared static _connectionFactory
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:000000047fd31438 <<
000007fee1ce8290 4001085 eb0 ...eAccessPermission 0 shared static ExecutePermission
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:000000047fd352f8 <<
000007fee1cf6408 400108a a08 System.Int32 1 shared static _objectTypeCount
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:322560 <<
Picked one Sql Connection object that is still open (_closeCount is zero!). Now let's make sure it's unreachable (i.e. leaked!).
0:079> !GCRoot 00000004c08c36c8
Thread 254c:
0000000014fbc2e0 000007fedf4f4229 System.Data.ProviderBase.DbConnectionPool.TryGetConnection(System.Data.Common.DbConnection, UInt32, Boolean, Boolean, System.Data.Common.DbConnectionOptions, System.Data.ProviderBase.DbConnectionInternal ByRef)
rbp+c8: 0000000014fbc3d8
-> 00000004c08c36c8 System.Data.SqlClient.SqlConnection
...
0000000014fbc6a0 000007fedbb6a55d Microsoft.TeamFoundation.Framework.Server.TeamFoundationSqlResourceComponent.Execute(ExecuteType, System.Data.CommandBehavior)
rbp+210: 0000000014fbc920
-> 00000004c08c2b00 Microsoft.TeamFoundation.WorkItemTracking.Server.DalSqlResourceComponent6
-> 00000004c08c36c8 System.Data.SqlClient.SqlConnection
Found 7 unique roots (run '!GCRoot -all' to see all roots).
Well, that SQL connection is still in-use. (not leaked.) Let's check another one.
0:079> !gcroot 00000004c0280050
Found 0 unique roots (run '!GCRoot -all' to see all roots).
Ok, so it's unreachable and open, it's therefore a leaked connection. We can tell more than just the connection string by dumping the inner connection:
0:079> !do 00000004c0280050
Name: System.Data.SqlClient.SqlConnection
MethodTable: 000007fedf566170
EEClass: 000007fedf3a8110
Size: 128(0x80) bytes
File: D:\Windows\Microsoft.Net\assembly\GAC_64\System.Data\v4.0_4.0.0.0__b77a5c561934e089\System.Data.dll
Fields:
MT Field Offset Type VT Attr Value Name
000007fee1cf3c38 4000198 8 System.Object 0 instance 0000000000000000 __identity
000007fee0d6f2e8 40002d2 10 ...ponentModel.ISite 0 instance 0000000000000000 site
000007fee0d7dbe0 40002d3 18 ....EventHandlerList 0 instance 00000004c02819b8 events
000007fee1cf3c38 40002d1 1b8 System.Object 0 shared static EventDisposed
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:000000047fd55f50 <<
000007fedf581510 40002c0 20 ...hangeEventHandler 0 instance 0000000000000000 _stateChangeEventHandler
000007fedf57e4c0 400107c 28 ...t.SqlDebugContext 0 instance 0000000000000000 _sdc
000007fee1cf4f30 400107d 70 System.Boolean 1 instance 0 _AsyncCommandInProgress
000007fedf5863e0 400107e 30 ...ent.SqlStatistics 0 instance 0000000000000000 _statistics
000007fee1cf4f30 400107f 71 System.Boolean 1 instance 0 _collectstats
000007fee1cf4f30 4001080 72 System.Boolean 1 instance 0 _fireInfoMessageEventOnUserErrors
000007fedfb0de48 4001081 38 ....Task, mscorlib]] 0 instance 0000000000000000 _currentCompletion
000007fedf5698b0 4001082 40 ...ent.SqlCredential 0 instance 0000000000000000 _credential
000007fee1cf3658 4001083 48 System.String 0 instance 00000004c027ff88 _connectionString
000007fedf56c0e0 4001086 50 ...ConnectionOptions 0 instance 000000047fd5ed68 _userConnectionOptions
000007fedf56df40 4001087 58 ...nnectionPoolGroup 0 instance 000000047fd5f090 _poolGroup
000007fedf56d730 4001088 60 ...onnectionInternal 0 instance 00000003c00425d0 _innerConnection
000007fee1cf6408 4001089 68 System.Int32 1 instance 0 _closeCount
000007fee1cf6408 400108b 6c System.Int32 1 instance 322129 ObjectID
000007fee1cf3c38 400107b ea0 System.Object 0 shared static EventInfoMessage
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:000000047fd31420 <<
000007fedf56a3c8 4001084 ea8 ...ConnectionFactory 0 shared static _connectionFactory
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:000000047fd31438 <<
000007fee1ce8290 4001085 eb0 ...eAccessPermission 0 shared static ExecutePermission
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:000000047fd352f8 <<
000007fee1cf6408 400108a a08 System.Int32 1 shared static _objectTypeCount
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:322560 <<
0:079> !do 00000003c00425d0
Name: System.Data.SqlClient.SqlInternalConnectionTds
MethodTable: 000007fedf56e8a8
EEClass: 000007fedf3d0cd0
Size: 296(0x128) bytes
File: D:\Windows\Microsoft.Net\assembly\GAC_64\System.Data\v4.0_4.0.0.0__b77a5c561934e089\System.Data.dll
Fields:
MT Field Offset Type VT Attr Value Name
000007fee1cf6408 40008b8 38 System.Int32 1 instance 869 _objectID
000007fee1cf4f30 40008bb 44 System.Boolean 1 instance 0 _allowSetConnectionString
000007fee1cf4f30 40008bc 45 System.Boolean 1 instance 1 _hidePassword
000007fedf567c68 40008bd 3c System.Int32 1 instance 1 _state
000007fee1cf0850 40008be 8 System.WeakReference 0 instance 00000003c0042768 _owningObject
000007fedf56bfc8 40008bf 10 ....DbConnectionPool 0 instance 000000047fd5f468 _connectionPool
000007fedf568ca8 40008c0 18 ...ctionPoolCounters 0 instance 000000047fd31478 _performanceCounters
000007fedf56a6c8 40008c1 20 ...ferenceCollection 0 instance 00000003c0047278 _referenceCollection
000007fee1cf6408 40008c2 40 System.Int32 1 instance 0 _pooledCount
000007fee1cf4f30 40008c3 46 System.Boolean 1 instance 0 _connectionIsDoomed
000007fee1cf4f30 40008c4 47 System.Boolean 1 instance 0 _cannotBePooled
000007fee1cf4f30 40008c5 48 System.Boolean 1 instance 0 _isInStasis
000007fee1cf0980 40008c6 50 System.DateTime 1 instance 00000003c0042620 _createTime
000007fef53a39d8 40008c7 28 ...tions.Transaction 0 instance 0000000000000000 _enlistedTransaction
000007fef53a39d8 40008c8 30 ...tions.Transaction 0 instance 0000000000000000 _enlistedTransactionOriginal
000007fee1cf6408 40008b7 6e0 System.Int32 1 shared static _objectTypeCount
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:969 <<
000007fedf56f5d8 40008b9 618 ...teChangeEventArgs 0 shared static StateChangeClosed
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:000000047fd36570 <<
000007fedf56f5d8 40008ba 620 ...teChangeEventArgs 0 shared static StateChangeOpen
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:000000047fd36588 <<
000007fedf56ae10 4001258 58 ...lConnectionString 0 instance 000000047fd5ed68 _connectionOptions
000007fee1cf4f30 4001259 88 System.Boolean 1 instance 0 _isEnlistedInTransaction
000007fee1cf77d0 400125a 60 System.Byte[] 0 instance 0000000000000000 _promotedDTCToken
000007fee1cf77d0 400125b 68 System.Byte[] 0 instance 0000000000000000 _whereAbouts
000007fee1cf3658 400125c 70 System.String 0 instance 00000003c00432b8 <CurrentDatabase>k__BackingField
000007fee1cf3658 400125d 78 System.String 0 instance 000000047fd5ee80 <CurrentDataSource>k__BackingField
000007fedfb22358 400125e 80 ...egatedTransaction 0 instance 0000000000000000 <DelegatedTransaction>k__BackingField
000007fedf56ccc0 400126e 90 ...GroupProviderInfo 0 instance 000000047fd5f0d8 _poolGroupProviderInfo
000007fedf569ad8 400126f 98 ...lClient.TdsParser 0 instance 00000003c0042850 _parser
000007fedf56dbb0 4001270 a0 ...lient.SqlLoginAck 0 instance 00000003c0043340 _loginAck
000007fedf5698b0 4001271 a8 ...ent.SqlCredential 0 instance 0000000000000000 _credential
000007fee1cf4f30 4001272 89 System.Boolean 1 instance 1 _fConnectionOpen
000007fee1cf4f30 4001273 8a System.Boolean 1 instance 1 _fResetConnection
000007fee1cf3658 4001274 b0 System.String 0 instance 00000003c00432b8 _originalDatabase
000007fee1cf3658 4001275 b8 System.String 0 instance 0000000000000000 _currentFailoverPartner
000007fee1cf3658 4001276 c0 System.String 0 instance 00000003c0043310 _originalLanguage
000007fee1cf3658 4001277 c8 System.String 0 instance 00000003c0043310 _currentLanguage
000007fee1cf6408 4001278 100 System.Int32 1 instance 8000 _currentPacketSize
000007fee1cf6408 4001279 104 System.Int32 1 instance 0 _asyncCommandCount
000007fee1cf3658 400127a d0 System.String 0 instance 00000002ffcd1420 _instanceName
000007fedf56ef90 400127b d8 ...ctionPoolIdentity 0 instance 000000047fd3a240 _identity
000007fedf56f4d8 400127c e0 ...Tds+SyncAsyncLock 0 instance 00000003c00426f8 _parserLock
000007fee1cf6408 400127d 108 System.Int32 1 instance -1 _threadIdOwningParserLock
000007fedf56a818 400127e e8 ...eoutErrorInternal 0 instance 00000003c0042780 timeoutErrorInternal
000007fee16612d0 400127f f0 ...rence, mscorlib]] 0 instance 0000000000000000 _preparedCommands
000007fee1cf0da8 4001280 110 System.Guid 1 instance 00000003c00426e0 _clientConnectionId
000007fedf5860d0 4001281 f8 ...lient.RoutingInfo 0 instance 0000000000000000 _routingInfo
0:079> !do 00000003c0042850
Name: System.Data.SqlClient.TdsParser
MethodTable: 000007fedf569ad8
EEClass: 000007fedf3cf318
Size: 160(0xa0) bytes
File: D:\Windows\Microsoft.Net\assembly\GAC_64\System.Data\v4.0_4.0.0.0__b77a5c561934e089\System.Data.dll
Fields:
MT Field Offset Type VT Attr Value Name
000007fee1cf6408 40014d6 70 System.Int32 1 instance 864 _objectID
000007fedf56dcc0 40014d8 8 ...ParserStateObject 0 instance 00000003c00428f0 _physicalStateObj
000007fedf56dcc0 40014d9 10 ...ParserStateObject 0 instance 0000000000000000 _pMarsPhysicalConObj
000007fedf589758 40014da 74 System.Int32 1 instance 2 _state
000007fee1cf3658 40014db 18 System.String 0 instance 000000047fd5ee80 _server
000007fee1cf4f30 40014dc 88 System.Boolean 1 instance 0 _fResetConnection
000007fee1cf4f30 40014dd 89 System.Boolean 1 instance 0 _fPreserveTransaction
000007fedf569618 40014de 20 ...ient.SqlCollation 0 instance 00000003c00432f8 _defaultCollation
000007fee1cf6408 40014df 78 System.Int32 1 instance 1252 _defaultCodePage
000007fee1cf6408 40014e0 7c System.Int32 1 instance 1033 _defaultLCID
000007fee1cf7528 40014e1 28 System.Text.Encoding 0 instance 000000047fd3e630 _defaultEncoding
000007fedf57b7e8 40014e3 80 System.Int32 1 instance 0 _encryptionOption
000007fedf58c368 40014e4 30 ...ternalTransaction 0 instance 0000000000000000 _currentTransaction
000007fedf58c368 40014e5 38 ...ternalTransaction 0 instance 0000000000000000 _pendingTransaction
000007fee1ce2060 40014e6 68 System.Int64 1 instance 0 _retainedTransactionId
000007fee1cf6408 40014e7 84 System.Int32 1 instance 0 _nonTransactedOpenResultCount
000007fedf56e8a8 40014e8 40 ...rnalConnectionTds 0 instance 00000003c00425d0 _connHandler
000007fee1cf4f30 40014e9 8a System.Boolean 1 instance 0 _fMARS
000007fee1cf4f30 40014ea 8b System.Boolean 1 instance 0 _loginWithFailover
000007fee1cf0918 40014eb 48 ...ng.AutoResetEvent 0 instance 0000000000000000 _resetConnectionEvent
000007fedf584368 40014ec 50 ...ParserSessionPool 0 instance 0000000000000000 _sessionPool
000007fee1cf4f30 40014ed 8c System.Boolean 1 instance 1 _isShiloh
000007fee1cf4f30 40014ee 8d System.Boolean 1 instance 1 _isShilohSP1
000007fee1cf4f30 40014ef 8e System.Boolean 1 instance 1 _isYukon
000007fee1cf4f30 40014f0 8f System.Boolean 1 instance 1 _isKatmai
000007fee1cf4f30 40014f1 90 System.Boolean 1 instance 0 _isDenali
000007fee1cf77d0 40014f2 58 System.Byte[] 0 instance 00000003c0042be8 _sniSpnBuffer
000007fedf5863e0 40014f3 60 ...ent.SqlStatistics 0 instance 0000000000000000 _statistics
000007fee1cf4f30 40014f4 91 System.Boolean 1 instance 0 _statisticsIsInTransaction
000007fee1cf4f30 40014fb 92 System.Boolean 1 instance 0 _asyncWrite
000007fee1cf6408 40014d5 a40 System.Int32 1 shared static _objectTypeCount
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:964 <<
000007fee1ce8c48 40014d7 1100 ...eading.Tasks.Task 0 shared static completedTask
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:0000000000000000 <<
000007fedf57b7e8 40014e2 a44 System.Int32 1 shared static _sniSupportedEncryptionOption
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:0 <<
000007fee1cf77d0 40014f5 1108 System.Byte[] 0 shared static s_nicAddress
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:000000047fd3de68 <<
000007fee1cf4f30 40014f6 a4c System.Boolean 1 shared static s_fSSPILoaded
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:1 <<
000007fee1cdffa0 40014f7 a48 System.UInt32 1 shared static s_maxSSPILength
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:12256 <<
000007fee1cf77d0 40014f8 1110 System.Byte[] 0 shared static s_longDataHeader
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:000000047fd3cf80 <<
000007fee1cf3c38 40014f9 1118 System.Object 0 shared static s_tdsParserLock
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:000000047fd3cfb8 <<
000007fee1cf77d0 40014fa 1120 System.Byte[] 0 shared static s_xmlMetadataSubstituteSequence
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:000000047fd3cfd0 <<
000007fedf3905c8 40014fc 1128 ...rd, System.Data]] 0 shared static __tvpEmptyValue
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:000000047fd3d038 <<
0:079> !do 00000003c00428f0
Name: System.Data.SqlClient.TdsParserStateObject
MethodTable: 000007fedf56dcc0
EEClass: 000007fedf3d09b8
Size: 424(0x1a8) bytes
File: D:\Windows\Microsoft.Net\assembly\GAC_64\System.Data\v4.0_4.0.0.0__b77a5c561934e089\System.Data.dll
Fields:
MT Field Offset Type VT Attr Value Name
000007fee1cf6408 40015a4 128 System.Int32 1 instance 864 _objectID
000007fedf569ad8 40015a5 8 ...lClient.TdsParser 0 instance 00000003c0042850 _parser
000007fedf569528 40015a6 10 ...lClient.SNIHandle 0 instance 00000003c0043150 _sessionHandle
000007fee1cf0850 40015a7 18 System.WeakReference 0 instance 00000003c0042a98 _owner
000007fee1cf6408 40015a8 12c System.Int32 1 instance 0 _activateCount
000007fee1cf6408 40015a9 130 System.Int32 1 instance 8 _inputHeaderLen
000007fee1cf6408 40015aa 134 System.Int32 1 instance 8 _outputHeaderLen
000007fee1cf77d0 40015ab 20 System.Byte[] 0 instance 00000003c0045300 _outBuff
000007fee1cf6408 40015ac 138 System.Int32 1 instance 8 _outBytesUsed
000007fee1cf77d0 40015ad 28 System.Byte[] 0 instance 00000003c00433a8 _inBuff
000007fee1cf6408 40015ae 13c System.Int32 1 instance 676 _inBytesUsed
000007fee1cf6408 40015af 140 System.Int32 1 instance 676 _inBytesRead
000007fee1cf6408 40015b0 144 System.Int32 1 instance 0 _inBytesPacket
000007fee1cf5ab8 40015b1 170 System.Byte 1 instance 3 _outputMessageType
000007fee1cf5ab8 40015b2 171 System.Byte 1 instance 1 _messageStatus
000007fee1cf5ab8 40015b3 172 System.Byte 1 instance 1 _outputPacketNumber
000007fee1cf4f30 40015b4 173 System.Boolean 1 instance 0 _pendingData
000007fee1cf4f30 40015b5 174 System.Boolean 1 instance 0 _fResetEventOwned
000007fee1cf4f30 40015b6 175 System.Boolean 1 instance 0 _fResetConnectionSent
000007fee1cf4f30 40015b7 176 System.Boolean 1 instance 0 _errorTokenReceived
000007fee1cf4f30 40015b8 177 System.Boolean 1 instance 0 _bulkCopyOpperationInProgress
000007fee1cf4f30 40015b9 178 System.Boolean 1 instance 0 _bulkCopyWriteTimeout
000007fedf56ece8 40015ba 30 ...lClient.SNIPacket 0 instance 00000003c0047258 _sniPacket
000007fedf56ece8 40015bb 38 ...lClient.SNIPacket 0 instance 00000001ffdc1038 _sniAsyncAttnPacket
000007fedf56e0c8 40015bc 40 ....WritePacketCache 0 instance 00000003c0042ab0 _writePacketCache
000007fedf387940 40015bd 48 ...et, System.Data]] 0 instance 00000003c0042af8 _pendingWritePackets
000007fee1cf3c38 40015be 50 System.Object 0 instance 00000003c0042b48 _writePacketLockObject
000007fee1cf21b0 40015bf 188 ...Services.GCHandle 1 instance 00000003c0042a78 _gcHandle
000007fee1cf6408 40015c0 148 System.Int32 1 instance 1 _pendingCallbacks
000007fee1ce2060 40015c1 100 System.Int64 1 instance 3600000 _timeoutMilliseconds
000007fee1ce2060 40015c2 108 System.Int64 1 instance 0 _timeoutTime
000007fee1cf4f30 40015c3 179 System.Boolean 1 instance 0 _attentionSent
000007fee1cf4f30 40015c4 17a System.Boolean 1 instance 0 _attentionReceived
000007fee1cf4f30 40015c5 17b System.Boolean 1 instance 0 _attentionSending
000007fee1cf4f30 40015c6 17c System.Boolean 1 instance 0 _internalTimeout
000007fee1ce2060 40015c7 110 System.Int64 1 instance 634759841647582060 _lastSuccessfulIOTime
000007fee1cbeb28 40015c8 58 System.Object[] 0 instance 00000003c0042b60 _securePasswords
000007fee1cf63a8 40015c9 60 System.Int32[] 0 instance 00000003c0042b90 _securePasswordOffsetsInBuffer
000007fee1cf4f30 40015ca 17d System.Boolean 1 instance 0 _cancelled
000007fee1cf6408 40015cb 14c System.Int32 1 instance -1 _allowObjectID
000007fee1cf4f30 40015cc 17e System.Boolean 1 instance 0 _hasOpenResult
000007fedf58c368 40015cd 68 ...ternalTransaction 0 instance 0000000000000000 _executedUnderTransaction
000007fee1cd3ea8 40015ce 118 System.UInt64 1 instance 0 _longlen
000007fee1cd3ea8 40015cf 120 System.UInt64 1 instance 0 _longlenleft
000007fee1cf63a8 40015d0 70 System.Int32[] 0 instance 0000000000000000 _decimalBits
000007fee1cf77d0 40015d1 78 System.Byte[] 0 instance 00000004bff10550 _bTmp
000007fee1cf6408 40015d2 150 System.Int32 1 instance 0 _bTmpRead
000007fee1cd5ab0 40015d3 80 System.Text.Decoder 0 instance 0000000000000000 _plpdecoder
000007fee1cf4f30 40015d4 17f System.Boolean 1 instance 0 _accumulateInfoEvents
000007fedf39a538 40015d5 88 ...or, System.Data]] 0 instance 0000000000000000 _pendingInfoEvents
000007fee1cf77d0 40015d6 90 System.Byte[] 0 instance 00000003c0042bb0 _partialHeaderBuffer
000007fee1cf6408 40015d7 154 System.Int32 1 instance 0 _partialHeaderBytesRead
000007fedf56c2d8 40015d8 98 ...t._SqlMetaDataSet 0 instance 00000004c0284568 _cleanupMetaData
000007fedf5828e0 40015d9 a0 ...DataSetCollection 0 instance 0000000000000000 _cleanupAltMetaDataSetArray
000007fee1cf6408 40015da 158 System.Int32 1 instance 0 _tracePasswordOffset
000007fee1cf6408 40015db 15c System.Int32 1 instance 0 _tracePasswordLength
000007fee1cf6408 40015dc 160 System.Int32 1 instance 0 _traceChangePasswordOffset
000007fee1cf6408 40015dd 164 System.Int32 1 instance 0 _traceChangePasswordLength
000007fee1cf4f30 40015de 180 System.Boolean 1 instance 0 _receivedColMetaData
000007fedf571848 40015df 168 System.Int32 1 instance 11 _sniContext
000007fee1cf4f30 40015e0 181 System.Boolean 1 instance 0 _bcpLock
000007fedf56f070 40015e1 190 ...Object+NullBitmap 1 instance 00000003c0042a80 _nullBitmapInfo
000007fee1cfc0e0 40015e2 a8 ...bject, mscorlib]] 0 instance 0000000000000000 _networkPacketTaskSource
000007fee1cf1fe8 40015e3 b0 ...m.Threading.Timer 0 instance 0000000000000000 _networkPacketTimeout
000007fee1cf4f30 40015e4 182 System.Boolean 1 instance 1 _syncOverAsync
000007fee1cf4f30 40015e5 183 System.Boolean 1 instance 0 _snapshotReplay
000007fedf5729f0 40015e6 b8 ...ect+StateSnapshot 0 instance 0000000000000000 _snapshot
000007fee1cf1e50 40015e7 c0 ....ExecutionContext 0 instance 0000000000000000 _executionContext
000007fee1cf4f30 40015e8 184 System.Boolean 1 instance 0 _asyncReadWithoutSnapshot
000007fedf56c1b8 40015e9 c8 ...qlErrorCollection 0 instance 0000000000000000 _errors
000007fedf56c1b8 40015ea d0 ...qlErrorCollection 0 instance 0000000000000000 _warnings
000007fee1cf3c38 40015eb d8 System.Object 0 instance 00000003c0042bd0 _errorAndWarningsLock
000007fee1cf4f30 40015ec 185 System.Boolean 1 instance 0 _hasErrorOrWarning
000007fedf56c1b8 40015ed e0 ...qlErrorCollection 0 instance 0000000000000000 _preAttentionErrors
000007fedf56c1b8 40015ee e8 ...qlErrorCollection 0 instance 0000000000000000 _preAttentionWarnings
000007fee1cfc0e0 40015ef f0 ...bject, mscorlib]] 0 instance 0000000000000000 _writeCompletionSource
000007fee1cf6408 40015f0 16c System.Int32 1 instance 0 _asyncWriteCount
000007fee1cf3888 40015f1 f8 System.Exception 0 instance 0000000000000000 _delayedWriteAsyncCallbackException
000007fee1cf4f30 40015f2 186 System.Boolean 1 instance 0 _reading
000007fee1cf6408 40015a3 a54 System.Int32 1 shared static _objectTypeCount
>> Domain:Value 000000000179a3a0:NotInit 000000000ffb56e0:NotInit <<
_outBuff is our output buffer, i.e. what we sent to the server, and it has our command text... Now we know who's leaking!
0:079> !do 00000003c0045300
Name: System.Byte[]
MethodTable: 000007fee1cf77d0
EEClass: 000007fee16fa4b8
Size: 8024(0x1f58) bytes
Array: Rank 1, Number of elements 8000, Type Byte
Content: <...6............................G.e.t.C.o.n.s.t.a.n.t.S.e.t.s....@.p.a.r.t.i.t.i.o.n.I.d..&.......@.u.s.e.r.S.I.D...X.....4X.S>.-
Fields:
None