Add telemetry to your bot

APPLIES TO: SDK v4

Telemetry logging enables bot applications to send event data to telemetry services such as Application Insights. Telemetry offers insights into your bot by showing which features are used the most, detects unwanted behavior, and offers visibility into availability, performance, and usage.

This article describes how to implement telemetry in your bot using Application Insights. This article covers:

• The code required to wire up telemetry in your bot and connect to Application Insights.
• How to enable telemetry in your bot's Dialogs.
• How to enable telemetry to capture usage data from other services, like Azure AI services.
• How to visualize your telemetry data in Application Insights.

Important

For a regional bot that might collect personally identifiable information (PII) in telemetry, your Application Insights resource and your Azure Bot resource should be in the same region with the bot. If the resources are in different regions, the PII might leave the geographic region of the bot.

Prerequisites

C#

• The CoreBot sample code
• The Application Insights sample code
• A subscription to Microsoft Azure
• An Application Insights key
• Familiarity with Application Insights
• git

Note

The Application Insights sample code was built on top of the CoreBot sample code. This article will step you through modifying the CoreBot sample code to incorporate telemetry. If you're following along in Visual Studio, you'll have the Application Insights sample code by the time you're finished.

JavaScript

• The CoreBot sample code
• The Application Insights sample code
• A subscription to Microsoft Azure
• An Application Insights key
• Familiarity with Application Insights
• Visual Studio Code
• Node.js version 10.14 or later. Use command node --version to determine the version of node you have.
• Bot Framework Emulator

Note

The Application Insights sample code was built on top of the CoreBot sample code. This article will step you through modifying the CoreBot sample code to incorporate telemetry. If you're following along in Visual Studio Code, you'll have the Application Insights sample code by the time you're finished.

Enable telemetry in your bot

C#

This article starts from the CoreBot sample app and adds the code required to integrate telemetry into any bot. This will enable Application Insights to begin tracking requests.

Important

If you haven't setup your Application Insights account and created your Application Insights key, do that before proceeding.

1. Open the CoreBot sample app in Visual Studio.

2. Add the Microsoft.Bot.Builder.Integration.ApplicationInsights.Core NuGet package. For more information on using NuGet, see Install and manage packages in Visual Studio:

3. Include the following statements in Startup.cs:

   using Microsoft.ApplicationInsights.Extensibility;
   using Microsoft.Bot.Builder.ApplicationInsights;
   using Microsoft.Bot.Builder.Integration.ApplicationInsights.Core;
   using Microsoft.Bot.Builder.Integration.AspNet.Core;
   

   Tip

   If you're following along by updating the CoreBot sample code, you'll notice that the using statement for Microsoft.Bot.Builder.Integration.AspNet.Core already exists in the CoreBot sample.

4. Include the following code in the ConfigureServices() method in Startup.cs. This will make telemetry services available to your bot via dependency injection (DI):

   // This method gets called by the runtime. Use this method to add services to the container.
   public void ConfigureServices(IServiceCollection services)
   {
       ...
           // Create the Bot Framework Adapter with error handling enabled.
           services.AddSingleton<IBotFrameworkHttpAdapter, AdapterWithErrorHandler>();
   
           // Add Application Insights services into service collection
           services.AddApplicationInsightsTelemetry();
   
           // Create the telemetry client.
           services.AddSingleton<IBotTelemetryClient, BotTelemetryClient>();
   
           // Add telemetry initializer that will set the correlation context for all telemetry items.
           services.AddSingleton<ITelemetryInitializer, OperationCorrelationTelemetryInitializer>();
   
           // Add telemetry initializer that sets the user ID and session ID (in addition to other bot-specific properties such as activity ID)
           services.AddSingleton<ITelemetryInitializer, TelemetryBotIdInitializer>();
   
           // Create the telemetry middleware to initialize telemetry gathering
           services.AddSingleton<TelemetryInitializerMiddleware>();
   
           // Create the telemetry middleware (used by the telemetry initializer) to track conversation events
           services.AddSingleton<TelemetryLoggerMiddleware>();
       ...
   }
   

   Tip

   If you're following along by updating the CoreBot sample code, you'll notice that services.AddSingleton<IBotFrameworkHttpAdapter, AdapterWithErrorHandler>(); already exists.

5. Instruct the adapter to use the middleware code that was added to the ConfigureServices() method. You do this in AdapterWithErrorHandler.cs with the parameter TelemetryInitializerMiddleware telemetryInitializerMiddleware in the constructor's parameter list, and the Use(telemetryInitializerMiddleware); statement in the constructor as shown here:

       public AdapterWithErrorHandler(IConfiguration configuration, ILogger<BotFrameworkHttpAdapter> logger, TelemetryInitializerMiddleware telemetryInitializerMiddleware, ConversationState conversationState = null)
           : base(configuration, logger)
   {
       ...
       Use(telemetryInitializerMiddleware);
   }
   

6. You'll also need to add Microsoft.Bot.Builder.Integration.ApplicationInsights.Core to your list of using statements in AdapterWithErrorHandler.cs.

7. Add the Application Insights instrumentation key in your appsettings.json file. The appsettings.json file contains metadata about external services the bot uses while running. For example, Cosmos DB, Application Insights, and Azure AI services connection and metadata is stored there. The addition to your appsettings.json file must be in this format:

   {
       "MicrosoftAppId": "",
       "MicrosoftAppPassword": "",
       "ApplicationInsights": {
           "InstrumentationKey": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
       }
   }
   

   Note

   Details on getting the Application Insights instrumentation key can be found in the article Application Insights keys.

At this point, the preliminary work to enable telemetry using Application Insights is done. You can run your bot locally using the Emulator and then go into Application Insights to see what is being logged, such as response time, overall app health, and general running information.

Enable telemetry in your bot's dialogs

When adding a new dialog to any ComponentDialog, it will inherit the Microsoft.Bot.Builder.IBotTelemetryClient of its parent dialog. For example, in the CoreBot sample application, all dialogs are added to the MainDialog, which is a ComponentDialog. Once you set the TelemetryClient property to the MainDialog, all dialogs added to it will automatically inherit the telemetryClient from it, so it doesn't need to be explicitly set when adding dialogs.

Follow the steps below to update your CoreBot example:

1. In MainDialog.cs, update the constructor's parameter list to include the IBotTelemetryClient parameter, then set the MainDialog's TelemetryClient property to that value as shown in the following code snippet:

   public MainDialog(IConfiguration configuration, ILogger<MainDialog> logger, IBotTelemetryClient telemetryClient)
       : base(nameof(MainDialog))
   {
       // Set the telemetry client for this and all child dialogs.
       this.TelemetryClient = telemetryClient;
       ...
   }
   

Tip

If you're following along and updating the CoreBot sample code, you can refer to the Application Insights sample code if you run into any problems.

Telemetry is now added to your bot dialogs. If you run your bot now, you should see things being logged in Application Insights; however, if you have any integrated technology such as an Azure AI service, you'll need to add the TelemetryClient to that code as well.

JavaScript

This article starts with the CoreBot sample app and adds the code required to integrate telemetry into any bot. This will enable Application Insights to begin tracking requests.

Important

If you haven't setup your Application Insights account and created your Application Insights key, do that before proceeding.

1. Open the CoreBot sample app in Visual Studio Code.

2. Add the Application Insights key to your .env file: InstrumentationKey=<EnterInstrumentationKeyHere>. The .env file contains metadata about external services the bot uses while running. For example, Application Insights and Azure AI services connection and metadata is stored there. The addition to your .env file must be in this format:

   MicrosoftAppType=
   MicrosoftAppId=
   MicrosoftAppPassword=
   MicrosoftAppTenantId=
   LuisAppId=
   LuisAPIKey=
   LuisAPIHostName=
   InstrumentationKey=
   

   Note

   Details on getting the Application Insights instrumentation key can be found in the article Application Insights keys.

3. Add a reference to the modules ApplicationInsightsTelemetryClient and TelemetryInitializerMiddleware that are located in botbuilder-applicationinsights in the Bot Framework SDK. To do this, add the following code starting near the top of index.js, just after the code to import required packages:

   const { ApplicationInsightsTelemetryClient, TelemetryInitializerMiddleware } = require('botbuilder-applicationinsights');
   const { TelemetryLoggerMiddleware } = require('botbuilder-core');
   

   Tip

   The JavaScript Bot Samples use Node.js, which follows the CommonJS module system, and the built in require function to include modules that exist in separate files.

4. Create a new function at the end of index.js named getTelemetryClient that takes your instrumentation key as a parameter and returns a telemetry client using the ApplicationInsightsTelemetryClient module you previously referenced. This telemetry client is where your telemetry data will be sent to, in this case Application Insights.

   // Listen for Upgrade requests for Streaming.
   server.on('upgrade', async (req, socket, head) => {
       // Create an adapter scoped to this WebSocket connection to allow storing session data.
       const streamingAdapter = new CloudAdapter(botFrameworkAuthentication);
   
       // Set onTurnError for the CloudAdapter created for each connection.
       streamingAdapter.onTurnError = onTurnErrorHandler;
   

5. Next, you need to add the telemetry middleware to the adapter middleware pipeline. To do this, add the following code, starting just after the error handling code:

   
   // For local development, in-memory storage is used.
   // CAUTION: The Memory Storage used here is for local bot debugging only. When the bot
   // is restarted, anything stored in memory will be gone.
   const memoryStorage = new MemoryStorage();
   

6. In order for your dialog to report telemetry data, its telemetryClient must match the one used for the telemetry middleware, that is, dialog.telemetryClient = telemetryClient;

   const server = restify.createServer();
   server.listen(process.env.port || process.env.PORT || 3978, function() {
       console.log(`\n${ server.name } listening to ${ server.url }`);
       console.log('\nGet Bot Framework Emulator: https://aka.ms/botframework-emulator');
       console.log('\nTo talk to your bot, open the emulator select "Open Bot"');
   });
   

7. After creating the restify HTTP web server object, instruct it to use the bodyParser handler.

       }
       return new NullTelemetryClient();
   }
   

   Tip

   This uses the restify bodyParser function. restify is a "A Node.js web service framework optimized for building semantically correct RESTful web services ready for production use at scale. restify optimizes for introspection and performance, and is used in some of the largest Node.js deployments on Earth." For more information, see the restify web site.

   Node.js follows the CommonJS module system and uses the built-in require function to include modules that exist in separate files.

At this point, the preliminary work to enable telemetry using Application Insights is done. You can run your bot locally using the Emulator and then go into Application Insights to see what is being logged, such as response time, overall app health, and general running information.

Enable or disable activity event and personal information logging

C#

Enable or disable activity logging

By default, the TelemetryInitializerMiddleware will use the TelemetryLoggerMiddleware to log telemetry when your bot sends / receives activities. Activity logging creates custom event logs in your Application Insights resource. If you wish, you can disable activity event logging by setting logActivityTelemetry to false on the TelemetryInitializerMiddleware when registering it in Startup.cs.

public void ConfigureServices(IServiceCollection services)
{
    ...
    // Add the telemetry initializer middleware
    services.AddSingleton<TelemetryInitializerMiddleware>(sp =>
            {
                var loggerMiddleware = sp.GetService<TelemetryLoggerMiddleware>();
                return new TelemetryInitializerMiddleware(loggerMiddleware, logActivityTelemetry: false);
            });
    ...
}

Enable or disable personal information logging

By default, if activity logging is enabled, some properties on the incoming / outgoing activities are excluded from logging as they're likely to contain personal information, such as user name and the activity text. You can choose to include these properties in your logging by making the following change to Startup.cs when registering the TelemetryLoggerMiddleware.

public void ConfigureServices(IServiceCollection services)
{
    ...
    // Add the telemetry initializer middleware
    services.AddSingleton<TelemetryLoggerMiddleware>(sp =>
            {
                var telemetryClient = sp.GetService<IBotTelemetryClient>();
                return new TelemetryLoggerMiddleware(telemetryClient, logPersonalInformation: true);
            });
    ...
}

Next we'll see what needs to be included to add telemetry functionality to the dialogs. This will enable you to get additional information such as what dialogs run, and statistics about each one.

JavaScript

Enable or disable activity logging

By default, the TelemetryInitializerMiddleware will use the TelemetryLoggerMiddleware to log telemetry when your bot sends or receives activities. Activity logging creates custom event logs in your Application Insights resource. If you wish, you can disable activity event logging by setting logActivityTelemetry to false on the TelemetryInitializerMiddleware when registering it in index.js.

The following code snippet comes from sample 21.corebot-app-insights, and shows the call to TelemetryInitializerMiddleware:

// For local development, in-memory storage is used.
// CAUTION: The Memory Storage used here is for local bot debugging only. When the bot
// is restarted, anything stored in memory will be gone.
const memoryStorage = new MemoryStorage();

The code snippet below shows the change needed in sample 21.corebot-app-insights, in the call to TelemetryInitializerMiddleware to disable activity logging:

// Add telemetry middleware to the adapter middleware pipeline
var telemetryClient = getTelemetryClient(process.env.InstrumentationKey);
var telemetryLoggerMiddleware = new TelemetryLoggerMiddleware(telemetryClient);
var initializerMiddleware = new TelemetryInitializerMiddleware(telemetryLoggerMiddleware, false);
adapter.use(initializerMiddleware);

Enable or disable personal information logging

When activity logging is enabled, some properties on the incoming / outgoing activities are excluded from logging by default as they're likely to contain personal information, such as user name and the activity text. You can choose to include these properties in your logging by changing the logPersonalInformation parameter from false to true when registering the TelemetryLoggerMiddleware in index.js.

// For local development, in-memory storage is used.
// CAUTION: The Memory Storage used here is for local bot debugging only. When the bot
// is restarted, anything stored in memory will be gone.
const memoryStorage = new MemoryStorage();

Next we'll see what needs to be included to add telemetry functionality to the dialogs. This will enable you to get additional information such as what dialogs run, and statistics about each one.

Enabling telemetry to capture usage data from other services like LUIS and QnA Maker

Note

Azure AI QnA Maker will be retired on 31 March 2025. Beginning 1 October 2022, you won't be able to create new QnA Maker resources or knowledge bases. A newer version of the question and answering capability is now available as part of Azure AI Language.

Custom question answering, a feature of Azure AI Language, is the updated version of the QnA Maker service. For more information about question-and-answer support in the Bot Framework SDK, see Natural language understanding.

Note

Language Understanding (LUIS) will be retired on 1 October 2025. Beginning 1 April 2023, you won't be able to create new LUIS resources. A newer version of language understanding is now available as part of Azure AI Language.

Conversational language understanding (CLU), a feature of Azure AI Language, is the updated version of LUIS. For more information about language understanding support in the Bot Framework SDK, see Natural language understanding.

C#

We'll next implement telemetry functionality in your LUIS service. The LUIS service has built-in telemetry logging available so there's little you need to do to start getting telemetry data from LUIS. If you're interested in enabling telemetry in a QnA Maker enabled bot, see Add telemetry to your QnA Maker bot

1. The IBotTelemetryClient telemetryClient parameter is required in the FlightBookingRecognizer constructor in FlightBookingRecognizer.cs:

   public FlightBookingRecognizer(IConfiguration configuration, IBotTelemetryClient telemetryClient)
   

2. Next, enable the telemetryClient when you create your LuisRecognizer in the FlightBookingRecognizer constructor. Do this by adding the telemetryClient as a new LuisRecognizerOption:

   if (luisIsConfigured)
   {
       var luisApplication = new LuisApplication(
           configuration["LuisAppId"],
           configuration["LuisAPIKey"],
           "https://" + configuration["LuisAPIHostName"]);
   
       // Set the recognizer options depending on which endpoint version you want to use.
       var recognizerOptions = new LuisRecognizerOptionsV3(luisApplication)
       {
           TelemetryClient = telemetryClient,
       };
       _recognizer = new LuisRecognizer(recognizerOptions);
   }
   

That's it, you should have a functional bot that logs telemetry data into Application insights. You can use the Bot Framework Emulator to run your bot locally. You shouldn't see any changes in the bot's behavior, but it will be logging information into Application Insights. Interact with the bot by sending multiple messages and in the next section we'll review the telemetry results in Application Insights.

For information on testing and debugging your bot, you can refer to the following articles:

• Debug a bot
• Testing and debugging guidelines
• Debug with the Emulator

JavaScript

We'll next implement telemetry functionality in your LUIS service. The LUIS service has built-in telemetry logging available so there's little you need to do to start getting telemetry data from LUIS.

To enable the telemetry client in your LUIS recognizer:

1. Open FlightBookingRecognizer.js

2. Pass the telemetryClient parameter to the FlightBookingRecognizer constructor:

   constructor(config, telemetryClient) {
   

3. Set the telemetryClient field of the recognizerOptions object to the telemetryClient property that is passed into the FlightBookingRecognizer constructor, once done your constructor will appear as follows:

   if (luisIsConfigured) {
       // Set the recognizer options depending on which endpoint version you want to use e.g v2 or v3.
       // More details can be found in https://docs.microsoft.com/azure/cognitive-services/luis/luis-migration-api-v3
       const recognizerOptions = {
           apiVersion: 'v3',
           telemetryClient: telemetryClient
       };
   
       this.recognizer = new LuisRecognizer(config, recognizerOptions);
   }
   

4. And finally you need to include the telemetryClient when creating an instance of the FlightBookingRecognizer in index.js:

   const memoryStorage = new MemoryStorage();
   

That's it; you should have a functional bot that logs telemetry data into Application insights. You can use the Bot Framework Emulator to run your bot locally. You shouldn't see any changes in the bot's behavior, but it will be logging information into Application Insights. Interact with the bot by sending multiple messages, and the next section describes how to review the telemetry results in Application Insights.

For information on testing and debugging your bot, you can refer to the following articles:

• Debug a bot
• Testing and debugging guidelines
• Debug with the Emulator

Visualizing your telemetry data in Application Insights

Application Insights monitors the availability, performance, and usage of your bot application whether it's hosted in the cloud or on-premises. It uses the powerful data analysis platform in Azure Monitor to provide you with deep insights into your application's operations and diagnose errors without waiting for a user to report them. There are a few ways to see the telemetry data collected by Application Insights, two of the primary ways are through queries and the dashboard.

Querying your telemetry data in Application Insights using Kusto Queries

Use this section as a starting point to learn how to use log queries in Application Insights. It demonstrates two useful queries and provides links to other documentation with additional information.

To query your data

1. Go to the Azure portal

2. To go to your Application Insights page, select Monitor, then Applications, and find it there.

3. Once in your Application Insights, select Logs (Analytics).

   

4. This will bring up the Query window. Enter the following query and select Run:

   customEvents
   | where name=="WaterfallStart"
   | extend DialogId = customDimensions['DialogId']
   | extend InstanceId = tostring(customDimensions['InstanceId'])
   | join kind=leftouter (customEvents | where name=="WaterfallComplete" | extend InstanceId = tostring(customDimensions['InstanceId'])) on InstanceId
   | summarize starts=countif(name=='WaterfallStart'), completes=countif(name1=='WaterfallComplete') by bin(timestamp, 1d), tostring(DialogId)
   | project Percentage=max_of(0.0, completes * 1.0 / starts), timestamp, tostring(DialogId)
   | render timechart
   

5. This will return the percentage of waterfall dialogs that run to completion.

   

Tip

You can pin any query to your Application Insights dashboard by selecting the button on the top right of the Logs (Analytics) blade. Just select the dashboard you want it pinned to, and it will be available next time you visit that dashboard.

The Application Insights dashboard

Anytime you create an Application Insights resource in Azure, a new dashboard will automatically be created and associated with it. You can see that dashboard by selecting the button at the top of your Application Insights blade, labeled Application Dashboard.

Alternatively, to view the data, go to the Azure portal. Select Dashboard on the left, then select the dashboard you want from the drop-down.

There, you'll see some default information about your bot performance and any additional queries that you've pinned to your dashboard.

Additional Information

• Add telemetry to your QnA Maker bot
• What is Application Insights?
• Using Search in Application Insights
• Create custom KPI dashboards using Azure Application Insights