Finally Deploy and Audit LAPS with Project VAST, Part 2 of 2

Hi folks – and welcome to the second in a two-part series about the Visual Auditing Security Tool (VAST) and the Local Administrator Password Solution (LAPS) tool. In the first part of this series, we discussed credential theft vectors, examined how LAPS can effectively slow down attackers (and thereby raise the cost of attacking your organization), and discussed how VAST can help you quickly and effectively deploy LAPS.

In this entry, we’ll turn our attention away from deploying LAPS and toward the thornier issue of auditing LAPS. We’ll discuss why you should audit LAPS, look at some of the difficulties with effectively auditing LAPS, and closely examine how VAST can help empower your organization.

Why You Need LAPS Auditing (Hint: Because it allows you to use LAPS)

Here’s a scenario: A company, call it TailSpin Toys, has been carefully following the Security Privileged Access roadmap (https://aka.ms/privsec) for some time now. They’ve made good progress, carefully deploying LAPS, partitioning their credentials according to the tiered model, and implementing Privileged Access Workstations (PAWs).

In this new administrative model, highly privileged accounts (for Tier 0) become Domain Controller Admin accounts (rather than Domain Admin-style accounts). TailSpin Toys’ server administrators are thoughtfully segregated according to task, necessity, and role. And Tier 2, or Workstation Admins, are still…well…they’re still Workstation Admins. Helpdesk and deskside support techs authenticate to workstations using accounts with broad entitlements throughout the enterprise; these accounts often enjoy administrative access to wide swaths of machines in particular regions, or (in extreme cases) to machines across the entire enterprise.

See the problem? LAPS has dramatically reduced the attack surface of the built-in Administrator account on workstations. Privileged Access Workstations and credential partitioning has reduced the risk of credential seepage onto assume compromised machines that sit on the Interwebs. However (you knew it was coming, right?  😊), TailSpin Toys has inadvertently left behind a major attack vector. As an attacker, I can no longer go after the built-in Admin account, but I can win wide latitude for lateral account movement by simply stealing a helpdesk or deskside Admin (Tier 2) account. This is particularly easy to do, as Tier 2 admins typically don’t operate on PAWs, meaning their machines (and by extension, their credentials) are accessible to remote attackers. So an attacker potentially only has to compromise one (or a handful) of machines in order to win access to every machine that the Tier 2 administrators have access to.

So what exactly constitutes an easy, low-cost, low-disruption solution to this quandary? Embracing the use of LAPS-managed local admin accounts in place of domain-based ones for support calls, of course. This effectively solves the problem described, but presents a new challenge of its own. If Bob and Alice both have access to the store of LAPS passwords (granted by ACLs in AD), how can an organization practice trust but verify of who has done what with the local accounts? Now consider that many organizations wish to grant arbitrary access to an entire helpdesk organization (which could be dozens of people). You see the challenge?

Lucky for us, LAPS writes a nice audit trail each and every time a password is retrieved from AD. Unlucky for us, it can be challenging to use effectively.

The Challenge(s) of LAPS Auditing

To be fully transparent, this section should really be called “the three challenge(s) of LAPS auditing.” To understand each of these challenges, let’s first examine how exactly LAPS implements (or more accurately leverages) auditing in AD.

Assuming that auditing and System Access Control Lists (SACLs) are appropriately configured, each time an attribute value is read, AD writes a 4662 Event: An operation was performed on an object. To be more specific, it writes multiple 4662 events. Herein lies our first challenge.

In my lab, I performed a simple LAPS lookup with an account called DA-SA1 and successfully retrieved the LAPS password. Then I popped over to the Event Viewer to have a look at the audit trail. Right away, I discovered a problem you might have suspected all along: 4662 is a very chatty Event ID.

Right away, there’s a lot to sift through. In this case, I have the advantage of knowing the general time for the retrieval I wish to audit. Because this was a test, I know that I performed the retrieval during the 3:00 PM hour. Narrowing the audit log, I see five 4662s in the log during that time:

Our second challenge is that AD writes multiple 4662s for each LAPS retrieval. How can we tell which is the one we care about? Well, scrolling through the entries, I can play the game which of these is not like the others? And we have a winner.

Four of the 4662 entries are for the expected Read Property access. And one is for Control Access. Recall that MSMCSADMPWD is protected by its status as a confidential attribute. And recall that confidential attributes require not just Read access, but also Control access to retrieve.

We’re making progress – and there’s just one final hurdle to clear. When I’m working with organizations that have implemented LAPS, we almost always see applications, products or accounts making inefficient LDAP queries. An inefficient query is where an entity pulls back all of the attributes for an object or set of objects and then parses them in memory or in its own data store. As the name suggests, it’s inefficient and can also be very taxing on the LDAP server (in this case, AD). It’s bad practice, but it’s also very widespread. In this type of reality, the third challenge becomes clear. How do I know that the 4662 Control access is actually for MSMCSADMPWD? We know because the log tells us so long as we know where to look.

Directly underneath the Access Mask (0x100 for Control Access), we see a listing of Properties. These are objectGUIDs for the confidential attributes that have been accessed. One of these signifies our MSMCSADMPWD attribute. But which one?

I need to do some detective work. For this, we’ll use PowerShell to dump out the objectGUID for the MSMCSADMPWD attribute. Note that this is unique for each Forest.

So I know that this objectGUID (starting with 1dab and ending with ef26 is what we’re after. And we see it in the 4662 Control log:

We have successfully audited my example LAPS password retrieval.

Scaling the Solution

Unless you are the rare IT professional with time on his or her hands(!), it’s not realistic to go through this process to meaningfully audit each and every LAPS password retrieval. And unless your organization is willing to introduce shared passwords without audit-ability (which it should absolutely not do), we need to make this process repeatable. Data aggregation tools like Log Analytics (now part of Azure Monitor) can help us get started.

Because I know the AccessMask I need (0x100 for Control) and the objectGUID (which I retrieved earlier – the one that started with “1dab”), I could certainly craft a query off of those parameters:

This will go a long way toward making LAPS truly auditable. Yet, I’m still in the business of sifting through a lot of log files. It’s do-able, just not very efficient.

In Project VAST, we take this a step further – and control all of the queries and filtering in the background. Let’s have a look at the LAPS Audit tab.

Starting in the upper left-hand spot, we can view all of the accounts that have retrieved passwords in this data set (briandel, JJWatt and TomBrady). Clicking on one of them filters the dataset like so:

From the data at the bottom, a story begins to emerge. We see that account briandel was used to retrieve the password for the computer JONSH386VM. This is important for verification purposes: was there a change or problem ticket opened? Does this account belong to a person whose role it is to access this type of machine? Was the account owner of briandel at work during these hours? Is this typical behavior for this account? We can see the data and begin to make important decisions at a glance.

Further, we can start to baseline the expected behavior for each account.

I see this account was used to perform a total of 168 password retrievals. This is a deceptively important insight. Over time, an organization can and should baseline account behaviors. If briandel is used for retrievals in the ballpark of 170 times a week, then that should be what we come to expect. We can measure derivation and take appropriate action. If we notice that briandel has gone from around 170 retrievals down to only a few, we’ll need to ensure the account is being used appropriately and that another, more privileged one is not being used in its place. If it goes from around 170 to 270 or 1700, we know we may have an early indicator of compromise: the account may be being used to raid the AD store of LAPS passwords.

By visualizing data in the way we do with Project VAST, we can also utilize it in a more typical forensic fashion. Let’s say that we are concerned that something potentially injurious was done on a machine; maybe data was exfiltrated or configuration changes were made. We can filter by computer name and date/time.

Let’s say that the event in question happened at 2:30AM on October 31. By filtering to the computer in question and simply sorting the logs by retrieval time and date, I can see what happened:

I see that user JJWatt retrieved the password at 2:15AM; the owner of the account (indeed anyone with access to its credentials) must be part of my investigation.

By carefully filtering, sorting, and querying the 4662 data coming from AD, we can turn a potential liability into an easily audited solution.

That wraps it up for LAPS auditing and Project VAST. Stay tuned next month when we’ll discuss another security capability of visualizing security Big Data with Project VAST.

I invite you to follow me on Twitter @securitypfe. And until then, happy auditing.

- Jon