Share via

Developing Stream Analytics Jobs in Visual Studio

  • Article

Stream Analytics Tools for Visual Studio came out a couple months ago (here) and today I got a chance to try it out to prep some work for a project I am on.

After getting the tools and making sure I am ready to go I wanted to build and run the job locally, this required that I provide a local input file. My input in this case is stream from an event hub.

 

If you already setup an input points to an event in the stream analytics job using Azure Portal, you can use the Input Details blade in the portal to download sample data:

  1. Click on Sample Data button
  2. Select date range for data sample and click OK
  3. Wait ...
  4. Once you get the notification that data is ready, click on the notification and download the data file.

 

 

 

Or, you can use Service Bus Explorer to prepare an events data file based on real events coming into the event hub:

  1. Connect to Event Hub
  2. Create a listener on the consumer group, I created a consumer group just for this purpose
  3. On the events tab, select all events in the list
  4. Right click on the selected events and click "Save Selected Events"

 

 

 

 

Now I needed to prep the saved file in a format that ASA project will accept as a valid local input, for that I processed the content of the file into a new one using the following code:
string filePath = ConfigurationManager.AppSettings.Get("eventfile"); string jsonData = File.ReadAllText(filePath); var eventsArray = JArray.Parse(jsonData); var streamwriter = File.AppendText(ConfigurationManager.AppSettings.Get("outputfile")); foreach (var item in eventsArray) { streamwriter.WriteLine(item["body"]); } streamwriter.Flush(); streamwriter.Close();

 

Back to the Stream Analytics project in Visual Studio, Add a new item of type "Local Input" and point it to the output file with the processed content.

 

With this, I can develop and test stream analytics jobs locally using real data captured from event hub.