Share via

Part I - Monitoring IIS Logs with Operations Manager

  • Article

In this post I'll talk about  IIS log file monitoring.

Log file monitoring  and event collections rules are not  used as much as other features but in some cases  they provide   perfect solution for our monitoring needs.  In Part I we will create the required event collection rule and in Part 2 I’ll guide you on creating a custom report to present the events we collected.

 

Scenario

Customer is using  a web service to fetch some sensitive data  and wants to track who  is  querying the web service within the organization. Unfortunately this 3rd party web service  does not have any auditing capabilities so we will try to close this gap using OpsMgr.

I will use Orchestrator web service for this demo. same technique can be used to monitor any kind of  website  hosted in IIS.

 

IIS Logs

Before we dive into details we need to understand  the IIs log  format. IIs logs can be configured from  Internet Information Services Manager.

image

Here you can select the fields to be included in the log

image

 

Sample log file

This is a section from my orchestrator web service log.This is the log entry is created when you  browse a specific  runbook in orchestrator web console.

#Fields: date time s-sitename s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status time-taken

2014-10-12 20:53:19 W3SVC2 AZDEMOSRV1 fe80::b8b0:55b0:4ef:e23a%32 GET /Orchestrator2012/Orchestrator.svc/Jobs() $expand=Runbook,Statistics&$filter=startswith(Runbook%2FPath%2C'%5CSharepoint%5C')%20and%20(RunbookId%20eq%20guid'0b759245-c56e-4f41-83e7-6eded739ed99')&$select=Id,RunbookId……  81 DEMO\volkan fe80::b8b0:55b0:4ef:e23a%32 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.1…….) 200 0 0 9669

OpsMgr Event Collection Rules

Lets  start with creating an event collection rule. Navigate to Authoring pane in OpsMgr console  and create a new Event Collection rule. We will use Generic CV Text Log .

1

Give it a name  and select target . I’ll use  Windows Operations system as target and  create a disabled rule. We will  create an override to enable it on specific servers.

2

Specify the directory for the logs, log pattern and separator. In separator area just press space once , we will use space as our separator.

P.S If you have multiple websites in your IIS server make sure you specify the right directory.

3

Build event Expression Tab lets you to filter the  events using parameters. Before we can do this we need to now which parameter  represents the data we seek.  So lets have a look at he captured event

Event below is generated when we  start a runbook from Orchestrator web console

2014-10-12 19:12:56 W3SVC2 AZDEMOSRV1 fe80::b8b0:55b0:4ef:e23a%32 POST /Orchestrator2012/Orchestrator.svc/Jobs – 81 DEMO\volkan fe80::b8b0:55b0:4ef:e23a%32 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.1…) 201 0 0 1209

When this log is captured by OpsMgr System.ApplicationLog.GenericCSVLog.FilteredEventProvider will place each field under separate parameter as shown below.

<EventData>
  <DataItem type="System.ApplicationLog.GenericLogEntryData" time="2014-10-12T19:13:11.9659052+00:00" sourceHealthServiceId="FA577900-3B84-786B-8E5A-8B235CE22A2B">
    <LogFileDirectory>C:\inetpub\logs\LogFiles\W3SVC2</LogFileDirectory>
    <LogFileType>Generic CSV Log File Format</LogFileType>
    <LogFileName>C:\inetpub\logs\LogFiles\W3SVC2\u_ex141012.log</LogFileName>
    <Params>
      <Param>2014-10-12</Param>
      <Param>19:12:56</Param>
      <Param>W3SVC2</Param>
      <Param>AZDEMOSRV1</Param>
      <Param>fe80::b8b0:55b0:4ef:e23a%32</Param>
      <Param>POST</Param>
      <Param>/Orchestrator2012/Orchestrator.svc/Jobs</Param>
      <Param>-</Param>
      <Param>81</Param>
      <Param>DEMO\volkan</Param>
      <Param>fe80::b8b0:55b0:4ef:e23a%32</Param>
      <Param>Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.1…)</Param>
      <Param>201</Param>
      <Param>0</Param>
      <Param>0</Param>
      <Param>1209</Param>
    </Params>
  </DataItem>
</EventData>

Here I’m interested in in

Parameter 6   =   POST   this  filter will only collect  runbook start and stop log entries. When you specify parameters in Event expression you need to refer them as Params/Param[<number>]

 

4

After creating the rule I’ve created an override to enable this rule for specific servers

5

You can also use UNC share for this type rules , for more details check Kevin Holman's blog post

OperationsManagerDW Event Views

I like spending time on SQL management studio so I’ll  review the events from DW. You can use the query below to find the events captured by  Event Collection rules.  All CSV Text Log rules are published  as GenericCSVLog in data warehouse so  we will use this to filter  the captured events.

SELECT        TOP (1000) Event.vEvent.DateTime, vRule.RuleSystemName, Event.vEventDetail.EventData, vEventPublisher.EventPublisherName

FROM            Event.vEvent INNER JOIN
                         Event.vEventDetail ON Event.vEvent.EventOriginId = Event.vEventDetail.EventOriginId INNER JOIN
                         vEventLoggingComputer ON Event.vEvent.LoggingComputerRowId = vEventLoggingComputer.EventLoggingComputerRowId INNER JOIN
                         Event.vEventRule ON Event.vEvent.EventOriginId = Event.vEventRule.EventOriginId INNER JOIN
                         vRule ON Event.vEventRule.RuleRowId = vRule.RuleRowId INNER JOIN
                         vEventPublisher ON Event.vEvent.EventPublisherRowId = vEventPublisher.EventPublisherRowId
WHERE    vEventPublisher.EventPublisherName='GenericCSVLog'
--and    (vRule.RuleSystemName = 'MomUIGeneratedRulea7e8d1fadd4b48afb9bc433dd2a5decf')
ORDER BY Event.vEvent.DateTime DESC

Result;

image

You can review the events to find the rulesystemname for the rule you created and filter the query further by using the rulesystemname. Since we have created this in OpsMgr console  it has a automatically generated name. Clicking the eventdata will bring you all captured event details.

Now we have to use some XML querying in SQL to get what we need from these events.

SELECT        TOP (100) Event.vEvent.DateTime
,vEventDetail.EventData.value('(/EventData/DataItem/Params/Param)[10]', 'varchar(100)') as Username
,vEventDetail.EventData.value('(/EventData/DataItem/Params/Param)[6]', 'varchar(10)')+' - '+vEventDetail.EventData.value('(/EventData/DataItem/Params/Param)[7]', 'varchar(100)') as Operation
,vEventDetail.EventData.value('(/EventData/DataItem/Params/Param)[4]', 'varchar(100)') as Computername
,vEventDetail.EventData.value('(/EventData/DataItem/Params/Param)[8]', 'varchar(256)') as Details
,SUBSTRING(vEventDetail.EventData.value('(/EventData/DataItem/Params/Param)[8]', 'varchar(256)'),37,36) as JOBID

FROM            Event.vEvent INNER JOIN
                         Event.vEventDetail ON Event.vEvent.EventOriginId = Event.vEventDetail.EventOriginId INNER JOIN
                         vEventLoggingComputer ON Event.vEvent.LoggingComputerRowId = vEventLoggingComputer.EventLoggingComputerRowId INNER JOIN
                         Event.vEventRule ON Event.vEvent.EventOriginId = Event.vEventRule.EventOriginId INNER JOIN
                         vRule ON Event.vEventRule.RuleRowId = vRule.RuleRowId INNER JOIN
                         vEventPublisher ON Event.vEvent.EventPublisherRowId = vEventPublisher.EventPublisherRowId
WHERE    vEventPublisher.EventPublisherName='GenericCSVLog'
and    (vRule.RuleSystemName = 'MomUIGeneratedRulea7e8d1fadd4b48afb9bc433dd2a5decf')
ORDER BY Event.vEvent.DateTime DESC

Result

image

As you can see I’ve modified my rule to capture special get events as well to show you how to query and filter out XML fields.

SQL Reporting Services Report Access

Same technique can be used to  track  who is accessing the reports in SSRS .  You have to create another rule and use the following expression. first part tracks the report access from Reports   and second part tracks the usage when reports are run from reporting services ( in general SC products use second option to render report within their console)

P.S Check out Enable HTTP logging for SSRS

image

Eventdata shows the  who has run which report.

image

In PART 2  we will create  a custom report to display this information in  a user friendly way.

Comments