Share via

HTTP Data Collector API in a real customer scenario

  • Article

Hi there,

I am back writing a blog post about how I used the HTTP Data Collector API to fulfill a specific customer request.

Background:

My customer requested to create a computer group in OMS, whose membership was based on an attribute different from the computer name.

There was hence the need for a new custom attribute to be populated in OMS and to be later used in the Log Analytics to create a Computer Group.

Implementation:

Since the official documentation can be found on the HTTP Data Collector API TechNet page, I'll focus on the method I used to retrieve the data I need. Analyzing the PowerShell example on the TechNet page, I recognized 3 specific areas:

  1. On the top of the script there's an area containing your workspace access information. Here, you have to customize this info before running the script.
  2. In the center, there's the body or data part, used to retrieve your data. I am going to cover how to change this part in this post.
  3. In the bottom part, you see the area containing the functions used to sign in and submit the data. I am going to refer to this part as the "fixed part"

Because I need to query my data to create records with different dynamic values (coming from the query results) and it is not possible to use variables with the syntax used in the sample script (which is using fixed values), how do I convert into Json something that is using variables?

The solution I implemented, which consist in a small change of the original sample script, is using the following methods (add members, ConvertTo-Json) and is made of the following steps:

  1. Create the data structure
  2. Populate the data structure as needed
  3. Convert the data structure to Json

So, in the sample script below (attachments link in the bottom), you can see my script doing the following:

  1. Setting the Workspace ID and Primary Key
  2. Setting the LogType with my own type (see "My body part" green arrow)
  3. Retrieving the data to be sent to OMS (see "My body part" green arrow)
  4. Creating and setting the PowerShell object containing the data (see "My body part" green arrow)
  5. Converting the PowerShell Object to Json (see "My body part" green arrow)
  6. Sending the data (using the Build-Signature and Post-OMSData functions and the Post-OMSData call from the "fixed part")

Running the above script manually, will upload a record with the following format:

This is just a prototype of the script I left to the customer. Hence, it must be considered as an example to explain how to retrieve a registry key to be used as part of a Computer Group Log Search Query like this one:

RegistryKey_CL | where (Value_s == "BR1-Role") | distinct Computer

Lesson learned:

  1. LogType: It represents the information type used in the query. Just remind that OMS will add the "_CL" suffix to it.
  2. TimeGenerated: This one is not mandatory the default value is the data ingestion timestamp (the timestamp when the data get uploaded).
  3. Body part: by appropriately changing the body part you can create a script that uploads different information such as ActiveDirectory attributes (Description, EmployeeID, etc), Active Directory OU location, number of files in a given path and so on.
  4. Key fields in the record: Use it if you like to have a more detailed result from your query. Here's why, in this example, I included the computer name.

Just for the purpose of demonstrating that you can really change just the body part, I am also attaching another sample script which gets the OU Attribute for the local computer (if joined to AD).

That's all folks, for now. I will cover how to make the data submission automatic in another post.

Thanks,

Bruno.

HTTPApiCollectorActiveDirectoryOU.zip

HTTPApiCollectorRegKey.zip