Issue with Continuous Language Identification in Azure Speech SDK for Angular Application

Question

We are currently using the "microsoft-cognitiveservices-speech-sdk" in our Angular application (version 14) for speech transcription and translation. The transcription and translation functionality is working as expected.

However, we are encountering an issue while trying to implement the Continuous Language Identification feature for automatic language detection. Specifically, when we enable this feature, the system detects and transcribes only in the first detected language, without switching to other spoken languages during the session.

We were under the impression that the "Continuous Language Identification" feature would allow dynamic language detection throughout the transcription, but it seems to be restricted to the initial language it identifies.

Could you please assist us with:

1. Understanding if this is the expected behavior of the Continuous Language Identification feature.
2. Guidance on how to achieve continuous language detection and transcription for multiple languages in the same session.

Thank you for your assistance.

Answer 1

@sanjay.bisht For Jcontinuous language detection the property SpeechServiceConnection_LanguageIdMode should be set to Continuous.

As per documentation:

The SpeechServiceConnection_LanguageIdMode property is only required for continuous LID. Without it, the Speech service defaults to at-start LID. The supported values are AtStart for at-start LID or Continuous for continuous LID.

Also, for Speech to text or translation you must create a SpeechConfig from the wss://{region}.stt.speech.microsoft.com/speech/universal/v2 endpoint string i.e use fromEndpoint() method and then use setProperty() method to get the property SpeechServiceConnection_LanguageIdMode

Also, the limitations are Continuous LID doesn't support changing languages within the same sentence. For example, if you're primarily speaking Spanish and insert some English words, it doesn't detect the language change per word and Continuous LID is only supported with continuous recognition.

Try setting the above and check if it works, if you are seeing recognition only at start then it is probably using the default property value of AtStart for SpeechServiceConnection_LanguageIdMode,

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.