How to Manage Microsoft StreamInsight with PowerShell
Are you running the StreamInsight host (StreamInsightHost.exe) or do you have an application that hosts the StreamInsight engine? If so, you might wonder how an administrator could retrieve manageability information or access metadata in the StreamInsight engine. It is possible that you often cannot afford and do not want to develop a full-blown application in Microsoft Visual Studio every time just to look at diagnostic information. Also, scripting capabilities are crucial because they allow you to automate administration tasks.
In those scenarios, PowerShell is the tool of the trade – and it works very well for StreamInsight. PowerShell allows you to perform administration tasks without the need to write a full-blown application in Visual Studio – and typically it is also your preferred choice for building powerful scripts. Below is the sequence of steps in PowerShell 2.0 to pull diagnostic views from the StreamInsight host while running the ObjectModel sample application on the host. You can find the sample code based on the ObjectModel sample at the end of this post.
This already works with our CTP2 bits - enjoy!
PS C:\> [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.ComplexEventProcessing")
GAC Version Location
--- ------- --------
True v2.0.50727 C:\Windows\assembly\GAC_MSIL\Microsoft.ComplexEventProcessing\10.0.0.0__89845dcd8080cc91\Micro...
The first step above loads the StreamInsight client API into PowerShell. As you can see, StreamInsight is installed on the machine and we are picking up the assembly from the GAC.
PS C:\> $server = [Microsoft.ComplexEventProcessing.Server]::Connect("https://localhost/StreamInsight")
With this step, we are establishing a connection to the running StreamInsight host process (make sure you have it started) and keep the connection in our server object. Let’s now have a quick look at all the members of the server object:
PS C:\> $server | gm
TypeName: Microsoft.ComplexEventProcessing.Server
Name MemberType Definition
---- ---------- ----------
ClearDiagnosticSettings Method System.Void ClearDiagnosticSettings(System.Uri name)
CreateApplication Method Microsoft.ComplexEventProcessing.Application CreateApplication(string name)
CreateManagementService Method Microsoft.ComplexEventProcessing.ManagementService.IManagementService CreateManag...
Dispose Method System.Void Dispose()
Equals Method bool Equals(System.Object obj)
GetDiagnosticSettings Method Microsoft.ComplexEventProcessing.DiagnosticSettings GetDiagnosticSettings(System....
GetDiagnosticView Method Microsoft.ComplexEventProcessing.DiagnosticView GetDiagnosticView(System.Uri name)
GetHashCode Method int GetHashCode()
GetType Method type GetType()
SetDiagnosticSettings Method System.Void SetDiagnosticSettings(System.Uri name, Microsoft.ComplexEventProcessi...
ToString Method string ToString()
Applications Property System.Collections.Generic.IDictionary`2[[System.String, mscorlib, Version=2.0.0....
IsEmbedded Property System.Boolean IsEmbedded {get;}
Note the method for getting diagnostics views in the list above. Let’s invoke this to get event manager and plan manager statistics from the host:
PS C:\> $dv = $server.GetDiagnosticView("cep:/Server/EventManager")
PS C:\> $dv
Key Value
--- -----
All Events Count 19
All Events Memory (bytes) 249856
PS C:\> $dv = $server.GetDiagnosticView("cep:/Server/PlanManager")
PS C:\> $dv
Key Value
--- -----
Query Count 14
Stream Count 50
Operator Count 38
Since we are running the ObjectModel sample, we can also pull statistics for the traffic sensor query:
PS C:\> $dv = $server.GetDiagnosticView("cep:/Server/Application/ObjectModelSample/Query/TrafficSensorQuery")
PS C:\> $dv
Key Value
--- -----
Query State Suspended
Start Time 9/22/2009 5:34:02 PM
End Time 9/22/2009 5:34:03 PM
Stream Count 0
Operator Count 0
Total Incoming Event Count 553
Total Consumed Event Count 553
Total Produced Event Count 192
Total Outgoing Event Count 192
Last Incoming Event System Time 9/22/2009 5:34:02 PM
Last Consumed Event System Time 9/22/2009 5:34:02 PM
Last Produced Event System Time 9/22/2009 5:34:03 PM
Last Outgoing Event System Time 9/22/2009 5:34:03 PM
Total Consumed Events Latency (milliseconds) 14527.833
Total Produced Events Latency (milliseconds) 62457.0953
Total Outgoing Events Latency (milliseconds) 63553.2049
Last Produced CTI Timestamp 12/31/9999 11:59:59 PM
Stream Event Count 0
Stream Memory Including Events (bytes) 0
Operator Index Event Count 0
Operator Event Memory (bytes) 0
Operator Index Memory (bytes) 65870
Total Number of Times Operator Scheduled 708
Total Operator CPU Usage (milliseconds) 670
This diagnostic view lists lots of useful stuff. You can find a detailed discussion of the individual elements in the Operations topic of the StreamInsight help file. Some fields that I always look into are the event counts and in particular the last CTI timestamp. Those are key indicators of how lively my system is.
As mentioned above, this example assumes that the query is running in a StreamInsight server which exposes its Web Service endpoint. Starting from our samples, there are two ways to do this.
- Expose the Web Service of an embedded server. We will cover this in a separate blog posting.
- Use the StreamInsightHost.exe.
In order to run the Object Model sample against a separate a StreamInsight host instead of an embedded server, you need to tweak it a little bit. These steps are described in the README.txt that is enclosed in the samples package. Attached is the complete code of the file ObjectModel.cs with the necessary changes. Note that you will have to edit the paths of the input csv files in that code.