Capture Microsoft Azure Stream Analytics logs

Microsoft Azure Stream Analytics makes building real time solution very easy. Developers can build Stream Analytics job with few clicks.

While running Stream Analytics jobs, you may encounter an error which may cause Stream Analytics job to go in degrade/stop state. So it requires to figure out the error in job to troubleshoot it. There could be various error you might get into like improper data, issue at source/destination side, query computation fail etc. Before you start fixing the issue, it’s required to figure out the cause of the failing job.

There is a way to figure out the issue from audit logs in azure portal. During our work we figured out another way to capture logs for a Stream Analytics job. In this blog I am showcasing about various technique’s to capture Stream Analytics log including the one we came up using PowerShell.

Microsoft Azure Ibiza Portal (https://portal.azure.com)

• Click Audit log and click Filter. Change parameter to show only Stream analytic logs (as shown in picture below). Click Update.

• Now we’ll see logs for Azure Stream Analytics. if logs are more, we’ll have Load More option at the end to load more logs information.

Microsoft Azure Classic Portal (https://manage.windowsazure.com)

• Either you can go directly to management services (from left side pane) or click Operation Logs from Stream Analytics dashboard.

• Change parameters and click run to get all logs.

• If there are too many logs then you have to move from one page to another.

Capture Stream Analytics logs through PowerShell

There is another way to download Stream Analytics log which is from PowerShell. The PowerShell script not only capture the logs but also zip it in local system.

• Download PowerShell script from here
• Open Windows PowerShell ISE and copy script

• Run Script
• At first step, it will prompt for login information with which Azure subscription is linked. Stream Analytics job should be part of the subscription.

• Once authenticated successfully, it will showcase all the Stream Analytics jobs with in the subscription.
• Choose one or all jobs for which logs need to be extracted. Press Enter

• Provide number of days log need to be extracted. Press Enter

• After writing log files it will zip all content.

• Open windows explorer and look onto C:\Temp folder. There will be 1) StreamAnalytics folder 2) StreamAnalytics.zip file

• Open StreamAnalytics folder. Below are the files available in it.

Description of files.

• StreamAnalytics_JobXXXXXXXXXXXXXXXXXXX.txt and StreamAnalytics_JobXXXXXXXXXXXXXXXXXXX.json :- Both contains same information. Later file (with extension JSON) contains information in JSON format. The size of the file depends on number of jobs selected and number of days log required (max 14 days). It contains information like Subscription, Resource Name, Job name, date range, input, output and log.
• StreamAnalytics_JobsListing.txt :- This file contains information (mentioned below) about selected job.

I would like to thanks my colleagues (Jason Howell and Dan Shaver) for their contribution.

 

Hope it help in debugging Stream Analytics job.