Freigeben über

Use the new PowerShell cmdlet ConvertFrom-String to parse KLIST Kerberos ticket output

  • Artikel

Tired of hacking away at RegEx and string functions to parse text? This post is for you!

ConvertFrom-String

imageIn yesterday’s post we reviewed a simple example of the new PowerShell 5.x Convert-String cmdlet. Today we are going to study a complex example with ConvertFrom-String. This cmdlet was first documented here in a PowerShell team blog post. To make a long story short, the Microsoft Research team invented some fuzzy logic for parsing large quantities of text using sample data. The technology learns and adapts as it parses. If the output does not look correct, you simply add more examples demonstrating the variation that was not matched.

The trouble with KLIST

I was doing some Kerberos research lately, and I wanted to parse the output of KLIST to be more PowerShell friendly. KLIST displays current Kerberos tickets on a machine, but it is flat text. We will use ConvertFrom-String to turn it into beautiful, sortable, filterable object data like this in Out-Gridview:

image

First, let’s take a look at a sample of KLIST output:

 Current LogonId is 0:0x3f337

Cached Tickets: (12)

#0>    Client: Administrator @ CONTOSO.COM
 Server: krbtgt/CONTOSO.COM @ CONTOSO.COM
    KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
    Ticket Flags 0x60a10000 -> forwardable forwarded renewable pre_authent name_canonicalize 
    Start Time: 8/29/2016 16:06:22 (local)
  End Time:   8/30/2016 2:06:21 (local)
   Renew Time: 9/5/2016 16:06:21 (local)
   Session Key Type: AES-256-CTS-HMAC-SHA1-96
  Cache Flags: 0x2 -> DELEGATION 
  Kdc Called: 2012R2-DC.contoso.com

#1> Client: Administrator @ CONTOSO.COM
 Server: krbtgt/CONTOSO.COM @ CONTOSO.COM
    KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
    Ticket Flags 0x40e10000 -> forwardable renewable initial pre_authent name_canonicalize 
  Start Time: 8/29/2016 16:06:21 (local)
  End Time:   8/30/2016 2:06:21 (local)
   Renew Time: 9/5/2016 16:06:21 (local)
   Session Key Type: AES-256-CTS-HMAC-SHA1-96
  Cache Flags: 0x1 -> PRIMARY 
 Kdc Called: 2012R2-DC

#2> Client: Administrator @ CONTOSO.COM
 Server: host/2012R2-MS.contoso.com @ CONTOSO.COM
    KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
    Ticket Flags 0x40a10000 -> forwardable renewable pre_authent name_canonicalize 
  Start Time: 8/29/2016 16:12:25 (local)
  End Time:   8/30/2016 2:06:21 (local)
   Renew Time: 9/5/2016 16:06:21 (local)
   Session Key Type: AES-256-CTS-HMAC-SHA1-96
  Cache Flags: 0 
 Kdc Called: 2012R2-DC.contoso.com
...
...

One line of code!

The data contains multiple objects with multiple properties. Unfortunately that is all plain text. To parse this using regular expressions or string functions would be time-consuming and error-prone. Here is the ConvertFrom-String syntax used to produce the screenshot at the top of the article:

 klist | ConvertFrom-String -TemplateFile .\template.txt | Out-GridView

Template magic

Wow! Are you kidding?! Nope. That’s it. But… wait… what’s in that template.txt file? Ah. That is the magic. Let’s take a look:

 Current LogonId is 0:0xb4ffc69

Cached Tickets: (4)

#{[int]ID*:0}>  Client: {Client:Administrator @ CONTOSO.COM}
    Server: {Server:krbtgt/CONTOSO.COM @ CONTOSO.COM}
   KerbTicket Encryption Type: {KerbTicketEncryptionType:AES-256-CTS-HMAC-SHA1-96}
 Ticket Flags {TicketFlags:0x60a10000} -> {TicketFlagsEnum:forwardable forwarded renewable pre_authent name_canonicalize} 
    Start Time: {[datetime]StartTime:8/29/2016 16:06:22} (local)
    End Time:   {[datetime]EndTime:8/30/2016 2:06:21} (local)
   Renew Time: {[datetime]RenewTime:9/5/2016 16:06:21} (local)
 Session Key Type: {SessionKeyType:AES-256-CTS-HMAC-SHA1-96}
 Cache Flags: {CacheFlags:0x1} -> {CacheFlagsEnum:PRIMARY} 
   Kdc Called: {KdcCalled:2012R2-DC.contoso.com}

#{[int]ID*:1}>  Client: {Client:Administrator @ CORP.CONTOSO.COM}
   Server: {Server:krbtgt/CONTOSO.COM @ CONTOSO.COM}
   KerbTicket Encryption Type: {KerbTicketEncryptionType:AES-256-CTS-HMAC-SHA1-96}
 Ticket Flags {TicketFlags:0x40e10000} -> {TicketFlagsEnum:forwardable renewable initial pre_authent name_canonicalize} 
  Start Time: {[datetime]StartTime:8/29/2016 16:06:21} (local)
    End Time:   {[datetime]EndTime:8/30/2016 2:06:21} (local)
   Renew Time: {[datetime]RenewTime:9/5/2016 16:06:21} (local)
 Session Key Type: {SessionKeyType:AES-256-CTS-HMAC-SHA1-96}
 Cache Flags: {CacheFlags:0x2} -> {CacheFlagsEnum:DELEGATION} 
    Kdc Called: {KdcCalled:2012R2-DC.contoso.com}

#{[int]ID*:2}>  Client: {Client:Administrator @ CORP.NA.ALPINESKIHOUSE.COM}
 Server: {Server:host/2012R2-MS.contoso.com @ CONTOSO.COM}
   KerbTicket Encryption Type: {KerbTicketEncryptionType:AES-256-CTS-HMAC-SHA1-96}
 Ticket Flags {TicketFlags:0x40a10000} -> {TicketFlagsEnum:forwardable renewable pre_authent name_canonicalize} 
  Start Time: {[datetime]StartTime:8/29/2016 1:12:25} (local)
 End Time:   {[datetime]EndTime:8/30/2016 2:06:21} (local)
   Renew Time: {[datetime]RenewTime:9/5/2016 1:06:21} (local)
  Session Key Type: {SessionKeyType:AES-256-CTS-HMAC-SHA1-96}
 Cache Flags: {CacheFlags:0} 
    Kdc Called: {KdcCalled:2012R2-DC}

#{[int]ID*:3}>  Client: {Client:Administrator @ CONTOSO.COM}
    Server: {Server:RPCSS/2012R2-MS.contoso.com @ CONTOSO.COM}
  KerbTicket Encryption Type: {KerbTicketEncryptionType:RSADSI RC4-HMAC(NT)}
  Ticket Flags {TicketFlags:0x40a10000} -> {TicketFlagsEnum:forwardable renewable pre_authent name_canonicalize} 
  Start Time: {[datetime]StartTime:12/29/2016 16:12:25} (local)
   End Time:   {[datetime]EndTime:12/30/2016 12:06:21} (local)
 Renew Time: {[datetime]RenewTime:12/5/2016 16:06:21} (local)
    Session Key Type: {SessionKeyType:RSADSI RC4-HMAC(NT)}
  Cache Flags: {CacheFlags:0} 
    Kdc Called: {KdcCalled:2012R2-DC.contoso.com}

To create the parsing template we begin by capturing the KLIST output to a text file like this:

 KLIST > template.txt

Let’s break down the formatting in the template file:

  • Basic property parsing looks like this: { [datatype]PropertyName:DATA } . To extract meaningful data from flat text, you surround sample output data values with this syntax.
  • The datatype is optional, if you want to use strings for everything. I chose to use [int] and [datetime] for intelligent sorting and filtering of property data.
  • For multi-object parsing in a file like this, the first property on the object gets the asterisk * after the property name. In this case I chose ID to indicate a new record.

For each data instance in the template file we add some formatting to identify the properties we want to extract. We do not need to keep every instance of the certificates in the list, only enough to demonstrate different sample values. Study the example template above to find these differences:

  • Cache Flags
  • Cache Flags Enum (does not appear in every record)
  • KerbTicket Encryption Type
  • Session Key Type
  • Ticket Flags
  • Dates (single-digit vs. double-digit days/months)
  • Server short name vs. FQDN (fully qualified domain name)
  • FQDNs of varying dotted patterns
  • Etc.

Mash the easy button

Does that template.txt look difficult? It might. But it is actually not that bad, and it is MUCH easier than trying to write a RegEx or string functions to parse this. Actually, the fuzzy logic behind the cmdlet does exactly that. It studies each sample to identify differences (string length, spaces, capitalization, etc.) and then dynamically generates parsing code. Some consider this to be like AI (artificial intelligence) or ML (machine learning).

The template above took a couple hours of tweaking and experimenting. It works 95% of the time. Your mileage may vary. For example, you may need to tweak the template for international date formats or different type values.

Your turn…

I decided to keep this blog post short and leave other things for you to discover with the cmdlet. I have shown you one way to use it. There is more! Get-Help is your friend.

Now take this and go parse some of your own flat text use cases. Use the comments below to share your challenges and victories. Enjoy!

Comments

  • Anonymous
    September 14, 2016
    AwesomeThanks
  • Anonymous
    September 20, 2016
    The comment has been removed
  • Anonymous
    September 26, 2016
    comment
  • Anonymous
    December 01, 2016
    Great post, will try this asap to see it in action. Thx.
  • Anonymous
    January 25, 2017
    This is about 25x as slow as using grep and awk to pull just the values you need, in my measurements... are speed improvements in the works?PS ~> (0..10 | %{(measure-command {get-klist}).TotalMilliseconds} | measure -Average).Average3955.98753636364PS ~> (0..10 | %{(measure-command { klist 2>&1 | grep -m1 krbtgt | awk '{print $3, $4}'}).TotalMilliseconds} | measure -Average).Average155.061109090909
    • Anonymous
      February 22, 2017
      Use the -updateTemplate parameter on ConvertFrom-String to improve performance for future parsing. Just in my quick testing, future parsing attempts were anywhere from 50 to over 100 times faster.
  • Anonymous
    February 24, 2017
    Thanks for posting Ashley, very helpful indeedI love RegEx and the challenge of parsing data like this with RegEx to get the PowerShell properties I want, however now with this new cmdlet I will be able to get out in the sun a bit more often :)