ETW and logging recipes

See also: all the recipes and the intro

I have the other more detailed posts on the other aspects of the ETW logging, and here is a pile of assorted links and examples about it. Some very short introduction: The Windows logging has multiple layers. There are the pre-ETW Classic events (that can also be stored and interpreted by the ETW subsystem), the newer ETW events, and the even newer TraceLogging that does away with the manifests, instead embedding the manifest information into each event. The TraceLogging events are still ETW events and can be stored and interpreted by the ETW subsystem.

By the way, if you're looking for a way to interpret the ETW events, look for the library called "TDH".

Some of the tools mentioned here don't come in-box but need to be downloaded from MSDN.

There also are a couple of tools not mentioned in the recipes but that I feel should be mentioned: the Message Analyzer (downloadable from MSDN) and Setup And Boot Event Collector (an optional feature in Windows Server).

 

 # ETW references
# Exploring ETW
https://blogs.msdn.com/b/ntdebugging/archive/2009/09/08/exploring-and-decoding-etw-providers-using-event-log-channels.aspx
# Windows Events - general
https://msdn.microsoft.com/en-us/library/windows/desktop/aa964766(v=vs.85).aspx
# Writing an Instrumentation Manifest
https://msdn.microsoft.com/en-us/library/windows/desktop/dd996930(v=vs.85).aspx
# Accessing Remote Computers
https://msdn.microsoft.com/en-us/library/windows/desktop/dd996908(v=vs.85).aspx
# Consuming Events
https://msdn.microsoft.com/en-us/library/windows/desktop/aa363692(v=vs.85).aspx
# Processing Event Logs in PowerShell - old style!
https://msdn.microsoft.com/en-us/library/windows/desktop/aa363692(v=vs.85).aspx
# PowerShell events API, new style
https://technet.microsoft.com/en-us/library/dd367894.aspx
# ETW FAQ
https://social.msdn.microsoft.com/Forums/en-US/a1aa1350-41a0-4490-9ae3-9b4520aeb9d4/faq-common-questions-for-etw-and-windows-event-log?forum=etw
# Re-logger
https://msdn.microsoft.com/en-us/library/windows/desktop/hh706657(v=vs.85).aspx
# an example on how to write providers
https://msdn.microsoft.com/en-us/library/windows/desktop/aa364162(v=vs.85).aspx
# Instrumentation manifest for event publishers
https://msdn.microsoft.com/en-us/library/aa385619.aspx
# Example of a manifest with many elements
https://msdn.microsoft.com/en-us/library/gg154749.aspx

# ETW security and its autologger registry settings
https://www.geoffchappell.com/notes/windows/etw/security.htm

# Autologger
https://msdn.microsoft.com/en-us/library/windows/desktop/aa363687(v=vs.85).aspx
# GlobalLogger
https://msdn.microsoft.com/en-us/library/windows/desktop/aa363690(v=vs.85).aspx
# Example that creates a logger and collects data
https://msdn.microsoft.com/en-us/library/windows/desktop/ee441324%28v=vs.85%29.aspx

# Rendering an ETW event as XML
https://msdn.microsoft.com/en-us/library/windows/desktop/aa385768%28v=vs.85%29.aspx
# EvtRender()
https://msdn.microsoft.com/en-us/library/windows/desktop/aa385471%28v=vs.85%29.aspx
# Event schema
https://msdn.microsoft.com/en-us/library/windows/desktop/aa385201%28v=vs.85%29.aspx

# old-style event logging
https://msdn.microsoft.com/en-us/library/windows/desktop/aa363661%28v=vs.85%29.aspx
# new-style event logging
https://msdn.microsoft.com/en-us/library/windows/desktop/aa385780%28v=vs.85%29.aspx


# Writing an instrumentation manifest
https://msdn.microsoft.com/en-us/library/windows/desktop/dd996930%28v=vs.85%29.aspx
# RegisterEventSource()
https://msdn.microsoft.com/en-us/library/windows/desktop/aa363678%28v=vs.85%29.aspx
# SERVICE_STATUS
https://msdn.microsoft.com/en-us/library/windows/desktop/ms685996%28v=vs.85%29.aspx
# An installation example
https://msdn.microsoft.com/en-us/library/aa367563%28v=vs.85%29.aspx
# Manifest files reference
https://msdn.microsoft.com/en-us/library/aa375632%28v=vs.85%29.aspx

# TraceLogging API reference
https://msdn.microsoft.com/en-us/library/dn904637%28v=vs.85%29.aspx
# TraceLogging C++ quick start
https://msdn.microsoft.com/en-us/library/dn904627%28v=vs.85%29.aspx



# ETW events control
https://msdn.microsoft.com/en-us/magazine/cc163437.aspx
logman.exe
logman query providers # get the list of providers
wpr.exe -providers # get the list of providers that works on Nano
logman query -ets # get the list of sessions?
logman start "MyBootEvent" -o "c:\tmp\MyBootEvent.etl" -p "Microsoft-Windows-BootEvent-Collector" -ets # start saving to a file
logman stop "MyBootEvent" -ets # stop the recording, flushes the file
# In the Event Viewer, see in "Applications and Services Log/Microsoft/Windows/BootEvent-Collector"
# logman can also be used to collect the performance stats
logman.exe create counter %ComputerName%_30s_interval -f bincirc -v mmddhhmm -max 350 -c "\Cache\*" "\IPv4\*" "\LogicalDisk(*)\*" "\Memory\*" "\Netlogon\*" "\Network Interface(*)\*" "\Paging File(*)\*" "\Per Processor Network Activity Cycles(*)\*" "\Per Processor Network Interface Card Activity(*)\*" "\Processor(*)\*" "\Processor Information(*)\*" "\PhysicalDisk(*)\*" "\Process(*)\*" "\Physical Network Interface Card Activity(*)\*" "\Redirector\*" "\SMB Client Shares\*" "\SMB Server Shares(*)\*" "\SMB Server Sessions\*" "\Server\*" "\Server Work Queues(*)\*" "\System\*" "\TCPv4\*" -si 00:00:30 -s vm154s013791 -u Administrator *
# another tool present on NanoServer
wevtutil.exe
# Installing an ETW manifest (e.g. registering a provider)
wevtutil im lib\bevtcol.man # to use the default from the manifest
wevtutil im lib\bevtcol.man /rf:c:\Temp\bevtcol.exe /mf:c:\Temp\bevtcol.exe # to override the provider binary
# Uninstalling a manifest
wevtutil um lib\bevtcol.man
# Exporting an installed manifest back to a file, the manifest is found by the events
tracerpt.exe -export manifest.man -l trace.etl
# tracerpt can also be used to dump the events from an ETL file

# a rather useless event dump tool:
tracefmt.exe

# Windows Event Collector
wecutil

# tracelog manual
https://msdn.microsoft.com/en-us/library/windows/hardware/ff552994%28v=vs.85%29.aspx
https://msdn.microsoft.com/en-us/library/windows/hardware/ff553012(v=vs.85).aspx
# Tracelog.exe examples
https://msdn.microsoft.com/en-us/library/windows/hardware/ff553026%28v=vs.85%29.aspx
# Tracelog switches
https://msdn.microsoft.com/en-us/library/ff553012%28v=vs.85%29.aspx
# Tracelog main page and download with WDK
https://msdn.microsoft.com/en-us/library/ff552994%28v=vs.85%29.aspx
# How to create the TMF files for parsing the Classic traces from PDB with tracepdb.exe
https://msdn.microsoft.com/en-us/library/windows/hardware/ff553922%28v=vs.85%29.aspx
# enabling the kernel trace events on the target
tracelog.exe -start -rt -kd -nonet -nodisk
tracelog.exe -addautologger -rt -kd -nonet -nodisk # to register as auto-logger
# The default logger is 'NT Kernel Logger'
# adding autologger:
tracelog.exe -addautologger -rt -kd -nonet -nodisk
# list the current sessions
tracelog.exe -l
# Remove a session
tracelog.exe -remove Kernel
# Default file location for tracelog
C:\windows\system32\Logfiles\WMI\NT Kernel Logger.etl

# Autologger logger session permissions
# see https://www.geoffchappell.com/notes/windows/etw/security.htm
Controlled by entries in HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\WMI\Security
Value names match the GUID of the session or provider
data in SECURITY_DESCRIPTOR_RELATIVE binary format

# printing WMI events in the windbg
!wmitrace.dynamicprint 1
# starting the trace
!wmitrace.start -kd ...
!wmitrace.kdtracing 1
# status
!wmitrace.strdump # list all loggers
!wmitrace.strdump 0x0 # status of logger 0x0

# Autologger is controlled by Registry, here is an example of NT Kernel Logger
reg add "HKLM\SYSTEM\CurrentControlSet\Control\WMI\Autologger\NT Kernel Logger" /f /v BufferSize /t REG_DWORD /d 0x40
reg add "HKLM\SYSTEM\CurrentControlSet\Control\WMI\Autologger\NT Kernel Logger" /f /v Guid /t REG_SZ /d "{9e814aad-3204-11d2-9a82-006008a86939}"
reg add "HKLM\SYSTEM\CurrentControlSet\Control\WMI\Autologger\NT Kernel Logger" /f /v Start /t REG_DWORD /d 1
reg add "HKLM\SYSTEM\CurrentControlSet\Control\WMI\Autologger\NT Kernel Logger" /f /v LogFileMode /t REG_DWORD /d 0x02880180
reg add "HKLM\SYSTEM\CurrentControlSet\Control\WMI\Autologger\NT Kernel Logger" /f /v EnableKernelFlags /t REG_BINARY /d 0100000000000000000000000000000000000000000000000000000000000000