Conversational language understanding (CLU), a feature of Azure AI Language, is the updated version of LUIS.
For more information about language understanding support in the Bot Framework SDK, see Natural language understanding.
The ability to understand what your user means conversationally and contextually can be a difficult task, but can provide your bot a more natural conversation feel. Language Understanding (LUIS) is a cloud-based API service that enables you to do just that so that your bot can recognize the intent of user messages, allow for more natural language from your user, and better direct the conversation flow.
This topic walks you through adding LUIS to a flight booking application to recognize different intents and entities contained within user input.
Note
The Bot Framework JavaScript, C#, and Python SDKs will continue to be supported, however, the Java SDK is being retired with final long-term support ending in November 2023.
Existing bots built with the Java SDK will continue to function.
This core bot sample shows an example of an airport flight booking application. It uses a LUIS service to recognize the user input and return the top recognized LUIS intent.
The language model contains three intents: Book Flight, Cancel, and None. LUIS will use these intents to understand what the user meant when they send a message to the bot. The language model also defines entities that LUIS can extract from the user's input, such as the origin or destination airport.
After each processing of user input, DialogBot saves the current state of both UserState and ConversationState. Once all the required information has been gathered, the coding sample creates a demo flight booking reservation. In this article, we'll be covering the LUIS aspects of this sample. However, the general flow of the sample is:
OnMembersAddedAsync is called when a new user is connected and displays a welcome card.
OnMessageActivityAsync is called for each user input received.
The OnMessageActivityAsync module runs the appropriate dialog through the Run dialog extension method. Then the main dialog calls the LUIS helper to find the top scoring user intent. If the top intent for the user input returns "BookFlight", the helper fills out information from the user that LUIS returned. After that, the main dialog starts the BookingDialog, which acquires additional information as needed from the user such as:
Origin the originating city
TravelDate the date to book the flight
Destination the destination city
After each processing of user input, dialogBot saves the current state of both userState and conversationState. Once all the required information has been gathered, the coding sample creates a demo flight booking reservation. In this article, we'll be covering the LUIS aspects of this sample. However, the general flow of the sample is:
onMembersAdded is called when a new user is connected and displays a welcome card.
OnMessage is called for each user input received.
The onMessage module runs the mainDialog, which gathers user input.
Then the main dialog calls the LUIS helper FlightBookingRecognizer to find the top scoring user intent. If the top intent for the user input returns "BookFlight", the helper fills out information from the user that LUIS returned.
Upon the response back, mainDialog preserves information for the user returned by LUIS and starts bookingDialog. bookingDialog acquires additional information as needed from the user such as
destination the destination city.
origin the originating city.
travelDate the date to book the flight.
After each processing of user input, DialogBot saves the current state of both UserState and ConversationState.
Once all the required information has been gathered, the coding sample creates a demo flight booking reservation.
In this article, we'll be covering the LUIS aspects of this sample. However, the general flow of the sample is:
onMembersAdded is called when a new user is connected and displays a welcome card.
onMessageActivity is called for each user input received.
The onMessageActivity module runs the appropriate dialog through the run dialog extension method. Then the main dialog calls the LUIS helper to find the top scoring user intent. If the top intent for the user input returns "BookFlight", the helper fills out information from the user that LUIS returned. After that, the main dialog starts the BookingDialog, which acquires additional information as needed from the user such as:
Origin the originating city
TravelDate the date to book the flight
Destination the destination city
After each processing of user input, DialogBot saves the current state of both user_state and conversation_state. Once all the required information has been gathered, the coding sample creates a demo flight booking reservation. In this article, we'll be covering the LUIS aspects of this sample. However, the general flow of the sample is:
on_members_added_activity is called when a new user is connected and displays a welcome card.
on_message_activity is called for each user input received.
The on_message_activity module runs the appropriate dialog through the run_dialog dialog extension method. Then the main dialog calls LuisHelper to find the top scoring user intent. If the top intent for the user input returns "BookFlight", the helper function fills out information from the user that LUIS returned. After that, the main dialog starts the BookingDialog, which acquires additional information as needed from the user such as:
On the Conversation apps page in LUIS, select Import, then Import as JSON.
In the Import new app dialog:
Choose the FlightBooking.json file in the CognitiveModels folder of the sample.
Enter FlightBooking as the optional name of the app, and select Done.
The site may display How to create an effective LUIS app and Upgrade your composite entities dialogs. You can dismiss these dialogs and continue.
Train your app, then publish your app to the production environment.
For more information, see the LUIS documentation on how to train and publish an app.
Why use entities
LUIS entities enable your bot to understand events beyond standard intents. This enables you to gather from users additional information, so your bot can ask questions and respond more intelligently. Along with definitions for the three LUIS intents 'Book Flight', 'Cancel', and 'None', the FlightBooking.json file also contains a set of entities such as 'From.Airport' and 'To.Airport'. These entities allow LUIS to detect and return additional information contained within the user's original input when they request a new travel booking.
Obtain values to connect to your LUIS app
Once your LUIS app is published, you can access it from your bot. You'll need to record several values to access your LUIS app from within your bot. You can retrieve that information using the LUIS portal.
Retrieve application information from the LUIS.ai portal
The settings file (appsettings.json, .env or config.py) acts as the place to bring all service references together in one place. The information you retrieve will be added to this file in the next section.
Add the information required to access your LUIS app including application ID, authoring key, and region into the appsettings.json file. In the previous step, you retrieved these values from your published LUIS app. The API host name should be in the format <your region>.api.cognitive.microsoft.com.
Add the information required to access your LUIS app including application ID, authoring key, and region into the .env file. In the previous step, you retrieved these values from your published LUIS app. The API host name should be in the format <your region>.api.cognitive.microsoft.com.
Add the information required to access your LUIS app including application ID, authoring key, and region into the application.properties file. In the previous step, you retrieved these values from your published LUIS app. The API host name should be in the format <your region>.api.cognitive.microsoft.com.
Add the information required to access your LUIS app including application ID, authoring key, and region into the config.py file. In the previous step, you retrieved these values from your published LUIS app. The API host name should be in the format <your region>.api.cognitive.microsoft.com.
Be sure that the Microsoft.Bot.Builder.AI.Luis NuGet package is installed for your project.
To connect to the LUIS service, the bot pulls the information you added to the appsetting.json file. The FlightBookingRecognizer class contains code with your settings from the appsetting.json file and queries the LUIS service by calling RecognizeAsync method.
FlightBookingRecognizer.cs
public class FlightBookingRecognizer : IRecognizer
{
private readonly LuisRecognizer _recognizer;
public FlightBookingRecognizer(IConfiguration configuration)
{
var luisIsConfigured = !string.IsNullOrEmpty(configuration["LuisAppId"]) && !string.IsNullOrEmpty(configuration["LuisAPIKey"]) && !string.IsNullOrEmpty(configuration["LuisAPIHostName"]);
if (luisIsConfigured)
{
var luisApplication = new LuisApplication(
configuration["LuisAppId"],
configuration["LuisAPIKey"],
"https://" + configuration["LuisAPIHostName"]);
// Set the recognizer options depending on which endpoint version you want to use.
// More details can be found in https://docs.microsoft.com/en-gb/azure/cognitive-services/luis/luis-migration-api-v3
var recognizerOptions = new LuisRecognizerOptionsV3(luisApplication)
{
PredictionOptions = new Bot.Builder.AI.LuisV3.LuisPredictionOptions
{
IncludeInstanceData = true,
}
};
_recognizer = new LuisRecognizer(recognizerOptions);
}
}
// Returns true if luis is configured in the appsettings.json and initialized.
public virtual bool IsConfigured => _recognizer != null;
public virtual async Task<RecognizerResult> RecognizeAsync(ITurnContext turnContext, CancellationToken cancellationToken)
=> await _recognizer.RecognizeAsync(turnContext, cancellationToken);
public virtual async Task<T> RecognizeAsync<T>(ITurnContext turnContext, CancellationToken cancellationToken)
where T : IRecognizerConvert, new()
=> await _recognizer.RecognizeAsync<T>(turnContext, cancellationToken);
}
The FlightBookingEx.cs contains the logic to extract From, To and TravelDate; it extends the partial class FlightBooking.cs used to store LUIS results when calling FlightBookingRecognizer.RecognizeAsync<FlightBooking> from the MainDialog.cs.
CognitiveModels\FlightBookingEx.cs
// Extends the partial FlightBooking class with methods and properties that simplify accessing entities in the luis results
public partial class FlightBooking
{
public (string From, string Airport) FromEntities
{
get
{
var fromValue = Entities?._instance?.From?.FirstOrDefault()?.Text;
var fromAirportValue = Entities?.From?.FirstOrDefault()?.Airport?.FirstOrDefault()?.FirstOrDefault();
return (fromValue, fromAirportValue);
}
}
public (string To, string Airport) ToEntities
{
get
{
var toValue = Entities?._instance?.To?.FirstOrDefault()?.Text;
var toAirportValue = Entities?.To?.FirstOrDefault()?.Airport?.FirstOrDefault()?.FirstOrDefault();
return (toValue, toAirportValue);
}
}
// This value will be a TIMEX. And we are only interested in a Date so grab the first result and drop the Time part.
// TIMEX is a format that represents DateTime expressions that include some ambiguity. e.g. missing a Year.
public string TravelDate
=> Entities.datetime?.FirstOrDefault()?.Expressions.FirstOrDefault()?.Split('T')[0];
}
To use LUIS, your project needs to install the botbuilder-ai npm package.
To connect to the LUIS service, the bot uses the information you added to the .env file. The flightBookingRecognizer.js class contains the code that imports your settings from the .env file and queries the LUIS service by calling recognize() method.
dialogs/flightBookingRecognizer.js
class FlightBookingRecognizer {
constructor(config) {
const luisIsConfigured = config && config.applicationId && config.endpointKey && config.endpoint;
if (luisIsConfigured) {
// Set the recognizer options depending on which endpoint version you want to use e.g v2 or v3.
// More details can be found in https://docs.microsoft.com/en-gb/azure/cognitive-services/luis/luis-migration-api-v3
const recognizerOptions = {
apiVersion: 'v3'
};
this.recognizer = new LuisRecognizer(config, recognizerOptions);
}
}
get isConfigured() {
return (this.recognizer !== undefined);
}
/**
* Returns an object with preformatted LUIS results for the bot's dialogs to consume.
* @param {TurnContext} context
*/
async executeLuisQuery(context) {
return await this.recognizer.recognize(context);
}
getFromEntities(result) {
let fromValue, fromAirportValue;
if (result.entities.$instance.From) {
fromValue = result.entities.$instance.From[0].text;
}
if (fromValue && result.entities.From[0].Airport) {
fromAirportValue = result.entities.From[0].Airport[0][0];
}
return { from: fromValue, airport: fromAirportValue };
}
getToEntities(result) {
let toValue, toAirportValue;
if (result.entities.$instance.To) {
toValue = result.entities.$instance.To[0].text;
}
if (toValue && result.entities.To[0].Airport) {
toAirportValue = result.entities.To[0].Airport[0][0];
}
return { to: toValue, airport: toAirportValue };
}
/**
* This value will be a TIMEX. And we are only interested in a Date so grab the first result and drop the Time part.
* TIMEX is a format that represents DateTime expressions that include some ambiguity. e.g. missing a Year.
*/
getTravelDate(result) {
const datetimeEntity = result.entities.datetime;
if (!datetimeEntity || !datetimeEntity[0]) return undefined;
const timex = datetimeEntity[0].timex;
if (!timex || !timex[0]) return undefined;
const datetime = timex[0].split('T')[0];
return datetime;
}
}
The logic to extract From, To and TravelDate is implemented as helper methods inside flightBookingRecognizer.js. These methods are used after calling flightBookingRecognizer.executeLuisQuery() from mainDialog.js
Be sure that the com.microsoft.bot.bot-ai-luis-v3 package is added to your pom.xml file.
To connect to the LUIS service, the bot pulls the information you added to the application.properties file. The FlightBookingRecognizer class contains code with your settings from the application.properties file and queries the LUIS service by calling recognize method.
FlightBookingRecognizer.java
/**
* The constructor of the FlightBookingRecognizer class.
*
* @param configuration The Configuration object to use.
*/
public FlightBookingRecognizer(Configuration configuration) {
Boolean luisIsConfigured = StringUtils.isNotBlank(configuration.getProperty("LuisAppId"))
&& StringUtils.isNotBlank(configuration.getProperty("LuisAPIKey"))
&& StringUtils.isNotBlank(configuration.getProperty("LuisAPIHostName"));
if (luisIsConfigured) {
LuisApplication luisApplication = new LuisApplication(
configuration.getProperty("LuisAppId"),
configuration.getProperty("LuisAPIKey"),
String.format("https://%s", configuration.getProperty("LuisAPIHostName"))
);
// Set the recognizer options depending on which endpoint version you want to use.
// More details can be found in
// https://docs.microsoft.com/en-gb/azure/cognitive-services/luis/luis-migration-api-v3
LuisRecognizerOptionsV3 recognizerOptions = new LuisRecognizerOptionsV3(luisApplication);
recognizerOptions.setIncludeInstanceData(true);
this.recognizer = new LuisRecognizer(recognizerOptions);
}
}
/**
* Runs an utterance through a recognizer and returns a generic recognizer result.
*
* @param turnContext Turn context.
* @return Analysis of utterance.
*/
@Override
public CompletableFuture<RecognizerResult> recognize(TurnContext turnContext) {
return this.recognizer.recognize(turnContext);
}
The FlightBookingRecognizer.cs contains the logic to extract From, To and TravelDate; and is called from the MainDialog.java to decode the results of the Luis query result.
FlightBookingRecognizer.java
/**
* Gets the From data from the entities which is part of the result.
*
* @param result The recognizer result.
* @return The object node representing the From data.
*/
public ObjectNode getFromEntities(RecognizerResult result) {
String fromValue = "", fromAirportValue = "";
if (result.getEntities().get("$instance").get("From") != null) {
fromValue = result.getEntities().get("$instance").get("From").get(0).get("text")
.asText();
}
if (!fromValue.isEmpty()
&& result.getEntities().get("From").get(0).get("Airport") != null) {
fromAirportValue = result.getEntities().get("From").get(0).get("Airport").get(0).get(0)
.asText();
}
ObjectMapper mapper = new ObjectMapper().findAndRegisterModules();
ObjectNode entitiesNode = mapper.createObjectNode();
entitiesNode.put("from", fromValue);
entitiesNode.put("airport", fromAirportValue);
return entitiesNode;
}
/**
* Gets the To data from the entities which is part of the result.
*
* @param result The recognizer result.
* @return The object node representing the To data.
*/
public ObjectNode getToEntities(RecognizerResult result) {
String toValue = "", toAirportValue = "";
if (result.getEntities().get("$instance").get("To") != null) {
toValue = result.getEntities().get("$instance").get("To").get(0).get("text").asText();
}
if (!toValue.isEmpty() && result.getEntities().get("To").get(0).get("Airport") != null) {
toAirportValue = result.getEntities().get("To").get(0).get("Airport").get(0).get(0)
.asText();
}
ObjectMapper mapper = new ObjectMapper().findAndRegisterModules();
ObjectNode entitiesNode = mapper.createObjectNode();
entitiesNode.put("to", toValue);
entitiesNode.put("airport", toAirportValue);
return entitiesNode;
}
/**
* This value will be a TIMEX. And we are only interested in a Date so grab the first result and
* drop the Time part. TIMEX is a format that represents DateTime expressions that include some
* ambiguity. e.g. missing a Year.
*
* @param result A {link RecognizerResult}
* @return The Timex value without the Time model
*/
public String getTravelDate(RecognizerResult result) {
JsonNode datetimeEntity = result.getEntities().get("datetime");
if (datetimeEntity == null || datetimeEntity.get(0) == null) {
return null;
}
JsonNode timex = datetimeEntity.get(0).get("timex");
if (timex == null || timex.get(0) == null) {
return null;
}
String datetime = timex.get(0).asText().split("T")[0];
return datetime;
}
Be sure that the botbuilder-ai PyPI package is installed for your project.
To connect to the LUIS service, the bot uses the information you added to the config.py file. The FlightBookingRecognizer class contains the code that imports your settings from the config.py file and queries the LUIS service by calling recognize() method.
flight_booking_recognizer.py
class FlightBookingRecognizer(Recognizer):
def __init__(self, configuration: DefaultConfig):
self._recognizer = None
luis_is_configured = (
configuration.LUIS_APP_ID
and configuration.LUIS_API_KEY
and configuration.LUIS_API_HOST_NAME
)
if luis_is_configured:
# Set the recognizer options depending on which endpoint version you want to use e.g v2 or v3.
# More details can be found in https://docs.microsoft.com/azure/cognitive-services/luis/luis-migration-api-v3
luis_application = LuisApplication(
configuration.LUIS_APP_ID,
configuration.LUIS_API_KEY,
"https://" + configuration.LUIS_API_HOST_NAME,
)
self._recognizer = LuisRecognizer(luis_application)
@property
def is_configured(self) -> bool:
# Returns true if luis is configured in the config.py and initialized.
return self._recognizer is not None
async def recognize(self, turn_context: TurnContext) -> RecognizerResult:
return await self._recognizer.recognize(turn_context)
The logic to extract From, To and travel_date is implemented as helper methods from the LuisHelper class inside luis_helper.py. These methods are used after calling LuisHelper.execute_luis_query() from main_dialog.py
helpers/luis_helper.py
class LuisHelper:
@staticmethod
async def execute_luis_query(
luis_recognizer: LuisRecognizer, turn_context: TurnContext
) -> (Intent, object):
"""
Returns an object with preformatted LUIS results for the bot's dialogs to consume.
"""
result = None
intent = None
try:
recognizer_result = await luis_recognizer.recognize(turn_context)
intent = (
sorted(
recognizer_result.intents,
key=recognizer_result.intents.get,
reverse=True,
)[:1][0]
if recognizer_result.intents
else None
)
if intent == Intent.BOOK_FLIGHT.value:
result = BookingDetails()
# We need to get the result from the LUIS JSON which at every level returns an array.
to_entities = recognizer_result.entities.get("$instance", {}).get(
"To", []
)
if len(to_entities) > 0:
if recognizer_result.entities.get("To", [{"$instance": {}}])[0][
"$instance"
]:
result.destination = to_entities[0]["text"].capitalize()
else:
result.unsupported_airports.append(
to_entities[0]["text"].capitalize()
)
from_entities = recognizer_result.entities.get("$instance", {}).get(
"From", []
)
if len(from_entities) > 0:
if recognizer_result.entities.get("From", [{"$instance": {}}])[0][
"$instance"
]:
result.origin = from_entities[0]["text"].capitalize()
else:
result.unsupported_airports.append(
from_entities[0]["text"].capitalize()
)
# This value will be a TIMEX. And we are only interested in a Date so grab the first result and drop
# the Time part. TIMEX is a format that represents DateTime expressions that include some ambiguity.
# e.g. missing a Year.
date_entities = recognizer_result.entities.get("datetime", [])
if date_entities:
timex = date_entities[0]["timex"]
if timex:
datetime = timex[0].split("T")[0]
result.travel_date = datetime
else:
result.travel_date = None
except Exception as exception:
print(exception)
return intent, result
LUIS is now configured and connected for your bot.
Run the sample locally on your machine. If you need instructions, refer to the README file for the C# Sample, JS Sample or Python Sample.
In the Emulator, type a message such as "travel to paris" or "going from paris to berlin". Use any utterance found in the file FlightBooking.json for training the intent "Book flight".
If the top intent returned from LUIS resolves to "Book flight", your bot will ask more questions until it has enough information stored to create a travel booking. At that point it will return this booking information back to your user.
At this point, the code bot logic will reset and you can continue to create more bookings.
