Natural language processing in Composer

APPLIES TO: Composer v1.x and v2.x

Natural language processing (NLP) is a technological process that enables computer applications, such as bots, to derive meaning from a user's input. To do this it attempts to identify valuable information contained in conversations by interpreting the user's needs (intents) and extract valuable information (entities) from a sentence, and respond back in a language the user will understand.

Why do bots need natural language processing?

Bots are able to provide little to no value without NLP. It's what enables your bot to understand the messages your users send and respond appropriately. When a user sends a message with "Hello", it's the bots Natural Language Processing capabilities that enable it to know that the user posted a standard greeting, which in turn allows your bot to leverage its AI capabilities to come up with a proper response. In this case, your bot can respond with a greeting.

Without NLP, your bot can't meaningfully differentiate between when a user enters "Hello" or "Goodbye". To a bot without NLP, "Hello" and "Goodbye" will be no different than any other string of characters grouped together in random order. NLP helps provide context and meaning to text or voice based user inputs so that your bot can come up with the best response.

One of the most significant challenges when it comes to NLP in your bot is the fact that users have a blank slate regarding what they can say to your bot. While you can try to predict what users will and won't say, there are bound to be conversations that you didn't anticipate, fortunately Bot Framework Composer makes it easy to continually refine its NLP capabilities.

There are four primary components of NLP in Composer:

• Language understanding: processes and interprets user input.
• Language generation (LG): produces bot responses.
• QnA: answer questions posed by bot users.
• Orchestrator: used to route user input.

Note

Azure QnA Maker will be retired on 31 March 2025. Beginning 1 October 2022, you won't be able to create new QnA Maker resources or knowledge bases. Language Understanding (LUIS) will be retired on 1 October 2025. Beginning 1 April 2023, you won't be able to create new LUIS resources. Newer versions of these services are now available as part of Azure AI Language.

Custom question answering, a feature of Azure AI Language, is the updated version of the QnA Maker service. Conversational language understanding (CLU), a feature of Azure AI Language, is the updated version of LUIS. For more information, see Natural language understanding in the Bot Framework SDK documentation.

Language Understanding

Language Understanding (LU) is the subset of NLP that deals with how the bot handles user inputs and converts them into something that it can understand and respond to intelligently.

Additional information on Language Understanding

• The Language Understanding concept article.
• The Advanced intent and entity definition concept article.
• The Using LUIS for Language Understanding how to article.

Language Generation

Language Generation (LG), is the process of producing meaningful phrases and sentences in the form of natural language—it's when your bot responds to a user with human readable language.

Additional information on Language Generation

• The Language Generation concept article.
• The Language Generation how to article.

QnA

Note

Azure QnA Maker will be retired on 31 March 2025. Beginning 1 October 2022, you won't be able to create new QnA Maker resources or knowledge bases. A newer version of the question and answering capability is now available as part of Azure AI Language.

Custom question answering, a feature of Azure AI Language, is the updated version of the QnA Maker service. For more information about question-and-answer support in Composer, see Natural language processing.

QnA is a subfield of NLP and information processing concerned with automatically answering questions posed by humans. Bots with QnA capabilities are able to recognize what a user is asking and respond/act appropriately.

Additional information about question answering

• The QnA how to article.
• The Create QnA Maker knowledge base article.
• About question answering support in the Bot Framework SDK.

Orchestrator

Orchestrators is an intent-only NLP engine. It's free software that doesn't require LUIS.

• The Orchestrator concept article.

Summary

Natural Language Processing is at the core of what most bots do in interpreting user's written or verbal inputs and responding to them in a meaningful way using a language they'll understand.

While NLP certainly can't work miracles and ensure a bot appropriately responds to every message, it's powerful enough to make-or-break a bot's success. Don't underestimate this critical and often overlooked aspect of bots.