Thread, System.Threading.Thread, and !Threads (III)
I got email asking me to explain !Threads output in details. I think this is a good question and a good topic for another installment to the series.
Here is an example I'll use for this post:
0:055> !threads
ThreadCount: 202
UnstartedThread: 95
BackgroundThread: 1
PendingThread: 0
DeadThread: 47
PreEmptive GC Alloc Lock
ID ThreadOBJ State GC Context Domain Count APT Exception
0 0xed0 0x0014f260 0x2000020 Enabled 0x00000000:0x00000000 0x00149aa0 1 Ukn
1 0xa3c 0x00157d28 0x2001220 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn (Finalizer)
XXX 0 0x00166378 0x1820 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
4 0x12cc 0x00166540 0x2001020 Enabled 0x00000000:0x00000000 0x00149aa0 0 STA
5 0x12dc 0x00166708 0x2001020 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
3 0xe7c 0x00175b70 0x2001020 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
XXX 0 0x00175d38 0x1820 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
XXX 0 0x00175f00 0x1820 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
XXX 0 0x001760c8 0x1820 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
XXX 0 0x00176290 0x1820 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn System.InvalidOperationException
XXX 0 0x00176458 0x1820 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
XXX 0 0x00176620 0x1820 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
XXX 0 0x001767e8 0x1820 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
12 0x7e0 0x001769b0 0x2001020 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
13 0x15e8 0x00178008 0x2001020 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
14 0x4d0 0x001781d0 0x2001020 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
XXX 0 0x00178398 0x1820 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
XXX 0 0x00178560 0x1820 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
XXX 0 0x00178728 0x1820 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
XXX 0 0x001788f0 0x1820 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
XXX 0 0x00178ab8 0x1820 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
XXX 0 0x00178c80 0x1820 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
XXX 0 0x00178e48 0x1820 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
21 0x14f0 0x00179010 0x2001020 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
22 0x1708 0x001791d8 0x2001020 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
23 0x11f8 0x001793a0 0x2001020 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
24 0x224 0x00179568 0x2001020 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
XXX 0 0x00179730 0x1820 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
XXX 0 0x001798f8 0x1820 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
XXX 0 0x00179ac0 0x1820 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn System.InvalidOperationException
XXX 0 0x00179c88 0x1820 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn System.InvalidOperationException
XXX 0 0x00179e50 0x1820 Enabled 0x00000000:0x00000000 0x00149aa0 0 Ukn
...
First !Threads gives some statistics about Thread Store.
ThreadCount: number of total C++ Thread objects in Thread Store.
UnstartedThread: number of C++ Thread objects marked as unstarted. Recall I mentioned in previous blog, if a user creates a C# Thread object, CLR will create an "unstarted" C++ Thread object. When Thread.Start is called on the C# object, CLR will create an OS thread and remove "unstarted" flag from the C++ Thread object.
BackgroundThread: number of C++ Threads (and the corresponding OS threads) considered as background. Being background simply means CLR won't wait the thread for shutting down. Threads created explicitly by using System.Threading.Thread.Start are by default foreground threads; whereas threads wandering into CLR from unmanaged world are by default background threads (Rotor: SetupThread in vm/threads.cpp calls SetBackground(TRUE) ). However, whether a thread is background could be changed by using IsBackground property in C# Thread object.
PendingThreads: If an OS thread is created but its ThreadProc hasn't be executed to the place to decrement unstarted counter in Thread Store, the thread is considered to be pending. Number of this type of threads should be quite low.
DeadThreads: Number of C++ Thread objects whose OS threads are already dead but the C++ objects themselves are not deleted yet.
In Rotor, all the five numbers are actually stored in ThreadStore (vm/threads.h) object as its fields.
Then it comes a table of all C++ Thread objects in Thread Store. Let me explain each field.
The first column doesn't have a header. It is the OS thread ID given by debugger just for debugging readability. Because the numbers only exist in debugger process, not the debuggee process, you may see the number being different when you look at a live session than when you debug a dump taken from the same live session. For a "dead" or "unstarted" thread, this column is "XXX".
ID: this is the thread ID assigned by OS, it remains consistent during debugger sessions, but OS could recycle it.
ThreadOBJ: address of C++ Thread object. You could see contents of the object by "dt mscorwks!Thread <address>" if you have symbols for mscorwks.dll.
State: one of the most important fields of the table. For Rotor, it is the C++ Thread's m_State field. It is combination of bit masks to indicate what the status the Thread currently is. All possible states (bit masks) are defined as enum ThreadState in vm/Threads. We already covered several states like TS_Background, TS_Unstarted, and TS_Dead. More states include TS_AbortRequested (this thread is requested to be aborted), TS_AbortInitiated (abort process is already started for this thread), TS_GCSuspendPending (GC is trying to suspend this thread), and etc.
Preemptive GC: also very important. In Rotor, this is m_fPreemptiveGCDisabled field of C++ Thread class. It indicates what GC mode the thread is in: "enabled" in the table means the thread is in preemptive mode where GC could preempt this thread at any time; "disabled" means the thread is in cooperative mode where GC has to wait the thread to give up its current work (the work is related to GC objects so it can't allow GC to move the objects around). When the thread is executing managed code (the current IP is in managed code), it is always in cooperative mode; when the thread is in Execution Engine (unmanaged code), EE code could choose to stay in either mode and could switch mode at any time; when a thread are outside of CLR (e.g, calling into native code using interop), it is always in preemptive mode.
GC Alloc context: allocate context GC might use when it tries to allocate object for this thread. In Rotor, it is m_alloc_context in C++ Thread object.
Domain: which AppDomain the thread is currently in (Rotor: m_pDomain field of C++ Thread class). You could use !DumpDomain or "dt mscorwks!AppDomain" to dump details of the domain. A thread can only be in one domain at a time, but it could switch into different domains. Speical marks will be put on thread's stack to when it transit to another domain.
Lock count: how many locks this thread has taken (Rotor: m_dwLockCount field of C++ Thread class). The locks it tracks include the managed monitors (taken by lock(obj) in C#), BCL's ReaderWriterLock, and certain locks inside CLR's unmanaged code.
APT: COM apartment for the thread, whether the thread is in a single-threaded apartment (STA), multithreaded apartment(MTA) or unknown.
Exception: the last managed exception thrown from this thread. It is saved in a GC handle in the C++ Thread object (Rotor: m_LastThrownObjectHandle).
The last column also indicates which special thread this thread is. However, !Threads only recognize a limited type of special threads for this field, including Finalizer thread, GC thread, Threadpool Worker thread, and Threadpool Completion Port thread. And for special threads which doesn't have a C++ Thread object (a special thread doesn't need to run managed code like debugger helper thread and server GC thread), they can not be displayed here. In Whidbey, a " -special" option is added to !Threads command which will show all special threads in the process as a separate list. Here is a sample output:
0:007> !threads -special
ThreadCount: 4
UnstartedThread: 0
BackgroundThread: 3
PendingThread: 0
DeadThread: 0
Hosted Runtime: no
PreEmptive GC Alloc Lock
ID OSID ThreadOBJ State GC Context Domain Count APT Exception
0 1 828 0029a030 a020 Disabled 06907c38:069081d4 0f59e038 2 MTA
4 2 16fc 0029e980 b220 Enabled 0690424c:069061d4 0021f4a8 0 MTA (Finalizer)
5 3 1c1c 002e71e8 1220 Enabled 028f20f8:028f3f94 0021f4a8 0 Ukn
6 4 1244 0f6fa778 80a220 Enabled 00000000:00000000 0021f4a8 0 MTA (Threadpool Completion Port)OSID Special thread type
1 e20 DbgHelper
2 1e1c GC
3 1ed4 GC
4 16fc Finalizer
5 1c1c ADUnloadHelper
6 1244 Timer
This posting is provided "AS IS" with no warranties, and confers no rights.
Comments
Anonymous
June 30, 2007
Stuff I've been intending to post a meaningful post about, but haven't: If you have ever wonderedAnonymous
February 10, 2008
In December I blogged about a little tool that i wrote to analyze hangs in dumps , and i showed the followingAnonymous
February 10, 2008
In December I blogged about a little tool that i wrote to analyze hangs in dumps , and i showed the followingAnonymous
February 10, 2008
PingBack from http://msdnrss.thecoderblogs.com/2008/02/11/hang-caused-by-gc-xml-deadlock/Anonymous
November 03, 2008
PingBack from http://debuggingblog.com/wp/2008/11/04/drwatson-minidump-with-windbg/