Muokkaa

Jaa

Enable Closed captions for Teams interoperability

  • Artikkeli

Learn how to allow your users to enable closed captions during a Teams interoperability scenario where your users might be in a meeting between an Azure Communication Services user and a Teams client user, or where your users are using Azure Communication Services calling SDK with their Microsoft 365 identity.

Prerequisites

Note

Please note that you will need to have a voice calling app using Azure Communication Services calling SDKs to access the closed captions feature that is described in this guide.

Models

Name Description
CaptionsCallFeature API for captions call feature
TeamsCaptions API for Teams captions
StartCaptionOptions Closed caption options like spoken language
TeamsCaptionsReceivedEventArgs Data object received for each Teams captions received event

Get closed captions feature

External Identity users and Microsoft 365 users

If you're building an application that allows Azure Communication Services users to join a Teams meeting

CaptionsCallFeature captionsCallFeature = call.Features.Captions;
CallCaptions callCaptions = await captionsCallFeature.GetCaptionsAsync();
if (callCaptions.CaptionsKind == CaptionsKind.TeamsCaptions)
{
    TeamsCaptions teamsCaptions = callCaptions as TeamsCaptions;
}

Subscribe to listeners

Add a listener to receive captions enabled/disabled status

teamsCaptions.CaptionsEnabledChanged += OnIsCaptionsEnabledChanged;

private void OnIsCaptionsEnabledChanged(object sender, PropertyChangedEventArgs args)
{
    if (teamsCaptions.IsEnabled)
    {
    }
}

Add listener for captions data received

teamsCaptions.CaptionsReceived += OnCaptionsReceived;

private void OnCaptionsReceived(object sender, TeamsCaptionsReceivedEventArgs eventArgs)
{
    // Information about the speaker.
    // eventArgs.Speaker
    // The original text with no transcribed.
    // eventArgs.SpokenText
    // language identifier for the captions text.
    // eventArgs.CaptionLanguage
    // language identifier for the speaker.
    // eventArgs.SpokenLanguage
    // The transcribed text.
    // eventArgs.CaptionText
    // Timestamp denoting the time when the corresponding speech was made.
    // eventArgs.Timestamp
    // CaptionsResultKind is Partial if text contains partially spoken sentence.
    // It is set to Final once the sentence has been completely transcribed.
    // eventArgs.ResultKind
}

Add a listener to receive active spoken language changed status

teamsCaptions.ActiveSpokenLanguageChanged += OnIsActiveSpokenLanguageChanged;

private void OnIsActiveSpokenLanguageChanged(object sender, PropertyChangedEventArgs args)
{
    // teamsCaptions.ActiveSpokenLanguage
}

Add a listener to receive active caption language changed status

teamsCaptions.ActiveCaptionLanguageChanged += OnIsActiveCaptionLanguageChanged;

private void OnIsActiveCaptionLanguageChanged(object sender, PropertyChangedEventArgs args)
{
    // teamsCaptions.ActiveCaptionLanguage
}

Start captions

Once you've set up all your listeners, you can now start adding captions.


private async void StartCaptions()
{
    var options = new StartCaptionsOptions
    {
        SpokenLanguage = "en-us"
    };
    try
    {
        await teamsCaptions.StartCaptionsAsync(options);
    }
    catch (Exception ex)
    {
    }
}

Stop captions

private async void StopCaptions()
{
    try
    {
        await teamsCaptions.StopCaptionsAsync();
    }
    catch (Exception ex)
    {
    }
}

Remove caption received listener

teamsCaptions.CaptionsReceived -= OnCaptionsReceived;

Spoken language support

Get list of supported spoken languages

Get a list of supported spoken languages that your users can select from when enabling closed captions.

// bcp 47 formatted language code
IReadOnlyList<string> sLanguages = teamsCaptions.SupportedSpokenLanguages;```

### Set spoken language 
When the user selects the spoken language, your app can set the spoken language that it expects captions to be generated from. 

``` cs 
public async void SetSpokenLanguage()
{
    try
    {
        await teamsCaptions.SetSpokenLanguageAsync("en-us");
    }
    catch (Exception ex)
    {
    }
}

Caption language support

Get supported caption language

If your organization has an active Teams premium license, then your Azure Communication Services users can enable translated captions as long as the organizer of the meeting has a Teams premium license. As for users with Microsoft 365 identities this check is done against their own user account if they meeting organizer doesn't have a Teams premium license.

// ISO 639-1 formatted language code
IReadOnlyList<string> cLanguages = teamsCaptions.SupportedCaptionLanguages;

Set caption language

public async void SetCaptionLanguage()
{
    try
    {
        await teamsCaptions.SetCaptionLanguageAsync("en");
    }
    catch (Exception ex)
    {
    }
}

Prerequisites

Note

Please note that you will need to have a voice calling app using Azure Communication Services calling SDKs to access the closed captions feature that is described in this guide.

Models

Name Description
CaptionsCallFeature API for Captions
CaptionsCommon Base class for captions
StartCaptionOptions Closed caption options like spoken language
TeamsCaptionHandler Callback definition for handling CaptionsReceivedEventType event
TeamsCaptionsInfo Data structure received for each CaptionsReceivedEventType event

Get closed captions feature

External Identity users

If you're building an application that allows Azure Communication Services users to join a Teams meeting.

let captionsCallFeature: SDK.CaptionsCallFeature = call.feature(SDK.Features.Captions);

Microsoft 365 users

let captionsCallFeature: SDK.CaptionsCallFeature = teamsCall.feature(SDK.Features.Captions);

Get teams captions object

You need to get and cast the Teams Captions object to utilize Teams Captions specific features

let teamsCaptions: SDK.TeamsCaptions;
if (captionsCallFeature.captions.kind === 'TeamsCaptions') {
    teamsCaptions = captionsCallFeature.captions as SDK.TeamsCaptions;
}

Subscribe to listeners

Add a listener to receive captions active/inactive status

const captionsActiveChangedHandler = () => {
    if (teamsCaptions.isCaptionsFeatureActive) {
        /* USER CODE HERE - E.G. RENDER TO DOM */
    }
}
teamsCaptions.on('CaptionsActiveChanged', captionsActiveChangedHandler);

Add a listener for captions data received

Handle the returned TeamsCaptionsInfo data object.

Note: The object contains a resultType prop that indicates whether the data is a partial caption or a finalized version of the caption. ResultType partial indicates live unedited caption, while final indicates a finalized interpreted version of the sentence (i.e includes punctuation and capitalization).

let currentCaptionLanguage : string;
const captionsReceivedHandler : TeamsCaptionsHandler = (data: TeamsCaptionsInfo) => { 
    /** USER CODE HERE - E.G. RENDER TO DOM 
     * data.captionLanguage
     * data.captionText
     * data.resultType
     * data.speaker
     * data.spokenText
     * data.timeStamp
    */
   // Example code:
   // Create a dom element, i.e. div, with id "captionArea" before proceeding with the sample code
    if (!this._currentCaptionLanguage || this._currentCaptionLanguage === data.captionLanguage) {
        let mri: string;
        switch (data.speaker.identifier.kind) {
            case 'communicationUser': { mri = data.speaker.identifier.communicationUserId; break; }
            case 'microsoftTeamsUser': { mri = data.speaker.identifier.microsoftTeamsUserId; break; }
            case 'phoneNumber': { mri = data.speaker.identifier.phoneNumber; break; }
        }
        const outgoingCaption = `prefix${mri.replace(/:/g, '').replace(/-/g, '')}`;

        let captionArea = document.getElementById("captionArea");
        const captionText = `${data.timestamp.toUTCString()}
            ${data.speaker.displayName}: ${data.captionText ?? data.spokenText}`;

        let foundCaptionContainer = captionArea.querySelector(`.${outgoingCaption}[isNotFinal='true']`);
        if (!foundCaptionContainer) {
            let captionContainer = document.createElement('div');
            captionContainer.setAttribute('isNotFinal', 'true');
            captionContainer.style['borderBottom'] = '1px solid';
            captionContainer.style['whiteSpace'] = 'pre-line';
            captionContainer.textContent = captionText;
            captionContainer.classList.add(newClassName);

            captionArea.appendChild(captionContainer);
        } else {
            foundCaptionContainer.textContent = captionText;

            if (captionData.resultType === 'Final') {
                foundCaptionContainer.setAttribute('isNotFinal', 'false');
            }
        }
    }
}; 
teamsCaptions.on('CaptionsReceived', captionsReceivedHandler);

Add a listener to receive spoken language changed status

const spokenLanguageChangedHandler = () => {
    if (teamsCaptions.activeSpokenLanguage !== currentSpokenLanguage) {
        /* USER CODE HERE - E.G. RENDER TO DOM */
    }
}
teamsCaptions.on('SpokenLanguageChanged', spokenLanguageChangedHandler)

Add a listener to receive caption language changed status

const captionLanguageChangedHandler = () => {
    if (teamsCaptions.activeCaptionLanguage !== currentCaptionLanguage) {
        /* USER CODE HERE - E.G. RENDER TO DOM */
    }
}
teamsCaptions.on('CaptionLanguageChanged', captionLanguageChangedHandler)

Start captions

Once you've set up all your listeners, you can now start adding captions.

try {
    await teamsCaptions.startCaptions({ spokenLanguage: 'en-us' });
} catch (e) {
    /* USER ERROR HANDLING CODE HERE */
}

Stop captions

try {
    teamsCaptions.stopCaptions(); 
} catch (e) {
    /* USER ERROR HANDLING CODE HERE */
}

Unsubscribe to listeners

teamsCaptions.off('CaptionsActiveChanged', captionsActiveChangedHandler);
teamsCaptions.off('CaptionsReceived', captionsReceivedHandler);

Spoken language support

Get a list of supported spoken languages

Get a list of supported spoken languages that your users can select from when enabling closed captions. The property returns an array of languages in bcp 47 format.

const spokenLanguages = teamsCaptions.supportedSpokenLanguages;

Set spoken language

Pass a value in from the supported spoken languages array to ensure that the requested language is supported. By default, if contoso provides no language or an unsupported language, the spoken language defaults to 'en-us'.

// bcp 47 formatted language code
const language = 'en-us'; 

// Altneratively, pass a value from the supported spoken languages array
const language = spokenLanguages[0]; 

try {
    teamsCaptions.setSpokenLanguage(language);
} catch (e) {
    /* USER ERROR HANDLING CODE HERE */
}

Caption language support

Get a list of supported caption languages

If your organization has an active Teams premium license, you can allow your users to use translated captions provided by Teams captions. As for users with a Microsoft 365 identity, if the meeting organizer doesn't have an active Teams premium license, captions language check is done against the Microsoft 365 users account.

The property returns an array of two-letter language codes in ISO 639-1 standard.

const captionLanguages = teamsCaptions.supportedCaptionLanguages;

Set caption language

// ISO 639-1 formatted language code
const language = 'en'; 

// Altneratively, pass a value from the supported caption languages array
const language = captionLanguages[0];
try {
    teamsCaptions.setCaptionLanguage(language);
} catch (e) {
    /* USER ERROR HANDLING CODE HERE */
}

Prerequisites

Note

Please note that you will need to have a voice calling app using Azure Communication Services calling SDKs to access the closed captions feature that is described in this guide.

Models

Name Description
CaptionsCallFeature API for captions call feature
TeamsCaptions API for Teams captions
StartCaptionOptions Closed caption options like spoken language
TeamsCaptionsListener Listener for TeamsCaptions addOnCaptionsReceivedListener
TeamsCaptionsReceivedEvent Data object received for each TeamsCaptionsListener event

Get closed captions feature

External Identity users and Microsoft 365 users

If you're building an application that allows users to join a Teams meeting

CaptionsCallFeature captionsCallFeature = call.feature(Features.CAPTIONS);
captionsCallFeature.getCaptions().whenComplete(
    ((captions, throwable) -> {
        if (throwable == null) {
            CallCaptions callCaptions = captions;
            if (captions.getCaptionsType() == CaptionsType.TEAMS_CAPTIONS) {
            // teams captions
            TeamsCaptions teamsCaptions = (TeamsCaptions) captions;
            }
        } else {
        // get captions failed
        // throwable is the exception/cause
        }
    }));

Subscribe to listeners

Add a listener to receive captions enabled/disabled status

public void addOnIsCaptionsEnabledChangedListener() {
    teamsCaptions.addOnCaptionsEnabledChangedListener( (PropertyChangedEvent args) -> {
        if(teamsCaptions.isEnabled()) {
            // captions enabled
        }
    });
}

Add listener for captions data received

TeamsCaptionsListener captionsListener = (TeamsCaptionsReceivedEvent args) -> {
  // Information about the speaker.
  // CallerInfo participantInfo = args.getSpeaker();
  // The original text with no transcribed.
  // args.getSpokenText();
  // language identifier for the captions text.
  // args.getCaptionLanguage();
  // language identifier for the speaker.
  // args.getSpokenLanguage();
  // The transcribed text.
  // args.getCaptionText();
  // Timestamp denoting the time when the corresponding speech was made.
  // args.getTimestamp();
  // CaptionsResultType is Partial if text contains partially spoken sentence.
  // It is set to Final once the sentence has been completely transcribed.
  // args.getResultType() == CaptionsResultType.FINAL;
}; 
public void addOnCaptionsReceivedListener() {
  teamsCaptions.addOnCaptionsReceivedListener(captionsListener); 
}

Add a listener to receive active spoken language changed status

public void addOnActiveSpokenLanguageChangedListener() {
    teamsCaptions.addOnActiveSpokenLanguageChangedListener( (PropertyChangedEvent args) -> {
       // teamsCaptions.getActiveSpokenLanguage()
    });
}

Add a listener to receive active caption language changed status

public void addOnActiveCaptionLanguageChangedListener() {
    teamsCaptions.addOnActiveCaptionLanguageChangedListener( (PropertyChangedEvent args) -> {
       // teamsCaptions.getActiveCaptionLanguage()
    });
}

Start captions

Once you've set up all your listeners, you can now start adding captions.

public void startCaptions() {
    StartCaptionsOptions startCaptionsOptions = new StartCaptionsOptions();
    startCaptionsOptions.setSpokenLanguage("en-us");
    teamsCaptions.startCaptions(startCaptionsOptions).whenComplete((result, error) -> {
        if (error != null) {
        }
    });
}

Stop captions

public void stopCaptions() {
    teamsCaptions.stopCaptions().whenComplete((result, error) -> {
        if (error != null) {
        }
    });
}

Remove caption received listener

public void removeOnCaptionsReceivedListener() {
    teamsCaptions.removeOnCaptionsReceivedListener(captionsListener);
}

Spoken language support

Get list of supported spoken languages

Get a list of supported spoken languages that your users can select from when enabling closed captions.

// bcp 47 formatted language code
teamsCaptions.getSupportedSpokenLanguages();

Set spoken language

When the user selects the spoken language, your app can set the spoken language that it expects captions to be generated from.

public void setSpokenLanguage() {
    teamsCaptions.setSpokenLanguage("en-us").whenComplete((result, error) -> {
        if (error != null) {
        }
    });
}

Caption language support

Get supported caption language

If your organization has an active Teams premium license, then your Azure Communication Services users can enable translated captions as long as the organizer of the meeting has a Teams premium license. As for users with Microsoft 365 identities this check is done against their own user account if they meeting organizer doesn't have a Teams premium license.

// ISO 639-1 formatted language code
teamsCaptions.getSupportedCaptionLanguages();

Set caption language

public void setCaptionLanguage() {
    teamsCaptions.setCaptionLanguage("en").whenComplete((result, error) -> {
        if (error != null) {
        }
    });
}

Prerequisites

Note

Please note that you will need to have a voice calling app using Azure Communication Services calling SDKs to access the closed captions feature that is described in this guide.

Models

Name Description
CaptionsCallFeature API for captions call feature
TeamsCaptions API for Teams captions
StartCaptionOptions Closed caption options like spoken language
TeamsCaptionsDelegate Delegate for Teams captions
TeamsCaptionsReceivedEventArgs Data object received for each Teams captions received event

Get closed captions feature

External Identity users and Microsoft 365 users

If you're building an application that allows Azure Communication Services users to join a Teams meeting

if let call = self.call {
    @State var captionsCallFeature = call.feature(Features.captions)
    captionsCallFeature.getCaptions{(value, error) in
        if let error = error {
            // failed to get captions
        } else {
            if (value?.type == CaptionsType.teamsCaptions) {
                // teams captions
                @State var teamsCaptions = value as? TeamsCaptions
            }
        }
    }
}

Subscribe to listeners

Add a listener to receive captions enabled/disabled, spoken language, caption language status changed and data received

extension CallObserver: TeamsCaptionsDelegate {
    // listener for receive captions enabled/disabled status
    public func teamsCaptions(_ teamsCaptions: TeamsCaptions, didChangeCaptionsEnabledState args: PropertyChangedEventArgs) {
        // teamsCaptions.isEnabled
    }
    
    // listener for active spoken language state change
    public func teamsCaptions(_ teamsCaptions: TeamsCaptions, didChangeActiveSpokenLanguageState args: PropertyChangedEventArgs) {
        // teamsCaptions.activeSpokenLanguage
    }
    
    // listener for active caption language state change
    public func teamsCaptions(_ teamsCaptions: TeamsCaptions, didChangeActiveCaptionLanguageState args: PropertyChangedEventArgs) {
        // teamsCaptions.activeCaptionLanguage
    }
    
    // listener for captions data received
    public func teamsCaptions(_ teamsCaptions: TeamsCaptions, didReceiveCaptions:TeamsCaptionsReceivedEventArgs) {
            // Information about the speaker.
            // didReceiveCaptions.speaker
            // The original text with no transcribed.
            // didReceiveCaptions.spokenText
            // language identifier for the captions text.
            // didReceiveCaptions.captionLanguage
            // language identifier for the speaker.
            // didReceiveCaptions.spokenLanguage
            // The transcribed text.
            // didReceiveCaptions.captionText
            // Timestamp denoting the time when the corresponding speech was made.
            // didReceiveCaptions.timestamp
            // CaptionsResultType is Partial if text contains partially spoken sentence.
            // It is set to Final once the sentence has been completely transcribed.
            // didReceiveCaptions.resultType
    }
}

teamsCaptions.delegate = self.callObserver

Start captions

Once you've set up all your listeners, you can now start adding captions.

func startCaptions() {
    guard let teamsCaptions = teamsCaptions else {
        return
    }
    let startCaptionsOptions = StartCaptionsOptions()
    startCaptionsOptions.spokenLanguage = "en-us"
    teamsCaptions.startCaptions(startCaptionsOptions: startCaptionsOptions, completionHandler: { (error) in
        if error != nil {
            
        }
    })
}

Stop captions

func stopCaptions() {
    teamsCaptions.stopCaptions(completionHandler: { (error) in
        if error != nil {
            
        }
    })
}

Remove caption received listener

teamsCaptions?.delegate = nil

Spoken language support

Get list of supported spoken languages

Get a list of supported spoken languages that your users can select from when enabling closed captions.

// bcp 47 formatted language code
let spokenLanguage : String = "en-us"
for language in teamsCaptions?.supportedSpokenLanguages ?? [] {
    // choose required language
    spokenLanguage = language
}

Set spoken language

When the user selects the spoken language, your app can set the spoken language that it expects captions to be generated from.

func setSpokenLanguage() {
    guard let teamsCaptions = self.teamsCaptions else {
        return
    }

    teamsCaptions.set(spokenLanguage: spokenLanguage, completionHandler: { (error) in
        if let error = error {
        }
    })
}

Caption language support

Get supported caption language

If your organization has an active Teams premium license, then your Azure Communication Services users can enable translated captions as long as the organizer of the meeting has a Teams premium license. As for users with Microsoft 365 identities this check is done against their own user account if they meeting organizer doesn't have a Teams premium license.

// ISO 639-1 formatted language code
let captionLanguage : String = "en"
for language in teamsCaptions?.supportedCaptionLanguages ?? [] {
    // choose required language
    captionLanguage = language
}

Set caption language

func setCaptionLanguage() {
    guard let teamsCaptions = self.teamsCaptions else {
        return
    }

    teamsCaptions.set(captionLanguage: captionLanguage, completionHandler: { (error) in
        if let error = error {
        }
    })
}

Clean up resources

If you want to clean up and remove a Communication Services subscription, you can delete the resource or resource group. Deleting the resource group also deletes any other resources associated with it. Learn more about cleaning up resources here.

Next steps

For more information, see the following articles: