Add speech to messages in the v3 JavaScript SDK

Article
12/13/2017

APPLIES TO: SDK v3

If you are building a bot for a speech-enabled channel such as Cortana, you can construct messages that specify the text to be spoken by your bot. You can also attempt to influence the state of the client's microphone by specifying an input hint to indicate whether your bot is accepting, expecting, or ignoring user input.

Specify text to be spoken by your bot

Using the Bot Framework SDK for Node.js, there are multiple ways to specify the text to be spoken by your bot on a speech-enabled channel. You can set the IMessage.speak property and send the message using the session.send() method, send the message using the session.say() method (passing parameters that specify display text, speech text, and options), or send the message using a built-in prompt (specifying options speak and retrySpeak).

IMessage.speak

If you are creating a message that will be sent using the session.send() method, set the speak property to specify the text to be spoken by your bot. The following code example creates a message that specifies text to be spoken and indicates that the bot is accepting user input.

var msg = new builder.Message(session)
    .speak('This is the text that will be spoken.')
    .inputHint(builder.InputHint.acceptingInput);
session.send(msg).endDialog();

session.say()

As an alternative to using session.send(), you can call the session.say() method to create and send a message that specifies the text to be spoken, in addition to the text to be displayed and other options. The method is defined as follows:

session.say(displayText: string, speechText: string, options?: object)

Parameter	Description
`displayText`	The text to be displayed.
`speechText`	The text (in plain text or SSML format) to be spoken.
`options`	An `IMessage` object that can contain an attachment or input hint.

The following code example sends a message that specifies text to be displayed and text to be spoken and indicates that the bot is ignoring user input.

session.say('Please hold while I calculate a response.', 
    'Please hold while I calculate a response.', 
    { inputHint: builder.InputHint.ignoringInput }
);

Prompt options

Using any of the built-in prompts, you can set the options speak and retrySpeak to specify the text to be spoken by your bot. The following code example creates a prompt that specifies text to be displayed, text to be spoken initially, and text to be spoken after waiting a while for user input. It indicates that the bot is expecting user input and uses SSML formatting to specify that the word "sure" should be spoken with a moderate amount of emphasis.

builder.Prompts.text(session, 'Are you sure that you want to cancel this transaction?', {            
    speak: 'Are you <emphasis level=\"moderate\">sure</emphasis> that you want to cancel this transaction?',
    retrySpeak: 'Are you <emphasis level=\"moderate\">sure</emphasis> that you want to cancel this transaction?',
    inputHint: builder.InputHint.expectingInput
});

Speech Synthesis Markup Language (SSML)

To specify text to be spoken by your bot, you can use either a plain text string or a string that is formatted as Speech Synthesis Markup Language (SSML), an XML-based markup language that enables you to control various characteristics of your bot's speech such as voice, rate, volume, pronunciation, pitch, and more. For details about SSML, see Speech Synthesis Markup Language Reference.

Tip

Use an SSML library to create well-formatted SSML.

Input hints

When you send a message on a speech-enabled channel, you can attempt to influence the state of the client's microphone by also including an input hint to indicate whether your bot is accepting, expecting, or ignoring user input. For more information, see Add input hints to messages.

Sample code

For a complete sample that shows how to create a speech-enabled bot using the Bot Framework SDK for Node.js, see the Roller sample in GitHub.

Share via