Quickstart: Create real-time diarization

Reference documentation | Package (NuGet) | Additional samples on GitHub

In this quickstart, you run an application for speech to text transcription with real-time diarization. Diarization distinguishes between the different speakers who participate in the conversation. The Speech service provides information about which speaker was speaking a particular part of transcribed speech.

The speaker information is included in the result in the speaker ID field. The speaker ID is a generic identifier assigned to each conversation participant by the service during the recognition as different speakers are being identified from the provided audio content.

Tip

You can try real-time speech to text in Speech Studio without signing up or writing any code. However, the Speech Studio doesn't yet support diarization.

Prerequisites

• An Azure subscription. You can create one for free.
• Create a Speech resource in the Azure portal.
• Get the Speech resource key and region. After your Speech resource is deployed, select Go to resource to view and manage keys.

Set up the environment

The Speech SDK is available as a NuGet package and implements .NET Standard 2.0. You install the Speech SDK later in this guide, but first check the SDK installation guide for any more requirements.

Set environment variables

You need to authenticate your application to access Azure AI services. This article shows you how to use environment variables to store your credentials. You can then access the environment variables from your code to authenticate your application. For production, use a more secure way to store and access your credentials.

Important

We recommend Microsoft Entra ID authentication with managed identities for Azure resources to avoid storing credentials with your applications that run in the cloud.

If you use an API key, store it securely somewhere else, such as in Azure Key Vault. Don't include the API key directly in your code, and never post it publicly.

For more information about AI services security, see Authenticate requests to Azure AI services.

To set the environment variables for your Speech resource key and region, open a console window, and follow the instructions for your operating system and development environment.

• To set the SPEECH_KEY environment variable, replace your-key with one of the keys for your resource.
• To set the SPEECH_REGION environment variable, replace your-region with one of the regions for your resource.

Windows

setx SPEECH_KEY your-key
setx SPEECH_REGION your-region

Note

If you only need to access the environment variables in the current console, you can set the environment variable with set instead of setx.

After you add the environment variables, you might need to restart any programs that need to read the environment variables, including the console window. For example, if you're using Visual Studio as your editor, restart Visual Studio before you run the example.

Linux

Bash

Edit your .bashrc file, and add the environment variables:

export SPEECH_KEY=your-key
export SPEECH_REGION=your-region

After you add the environment variables, run source ~/.bashrc from your console window to make the changes effective.

macOS

Bash

Edit your .bash_profile file, and add the environment variables:

export SPEECH_KEY=your-key
export SPEECH_REGION=your-region

After you add the environment variables, run source ~/.bash_profile from your console window to make the changes effective.

Xcode

For iOS and macOS development, you set the environment variables in Xcode. For example, follow these steps to set the environment variable in Xcode 13.4.1.

1. Select Product > Scheme > Edit scheme.
2. Select Arguments on the Run (Debug Run) page.
3. Under Environment Variables select the plus (+) sign to add a new environment variable.
4. Enter SPEECH_KEY for the Name and enter your Speech resource key for the Value.

To set the environment variable for your Speech resource region, follow the same steps. Set SPEECH_REGION to the region of your resource. For example, westus.

For more configuration options, see the Xcode documentation.

Implement diarization from file with conversation transcription

Follow these steps to create a console application and install the Speech SDK.

1. Open a command prompt window in the folder where you want the new project. Run this command to create a console application with the .NET CLI.

   dotnet new console
   

   This command creates the Program.cs file in your project directory.

2. Install the Speech SDK in your new project with the .NET CLI.

   dotnet add package Microsoft.CognitiveServices.Speech
   

3. Replace the contents of Program.cs with the following code.

   using Microsoft.CognitiveServices.Speech;
   using Microsoft.CognitiveServices.Speech.Audio;
   using Microsoft.CognitiveServices.Speech.Transcription;
   
   class Program 
   {
       // This example requires environment variables named "SPEECH_KEY" and "SPEECH_REGION"
       static string speechKey = Environment.GetEnvironmentVariable("SPEECH_KEY");
       static string speechRegion = Environment.GetEnvironmentVariable("SPEECH_REGION");
   
       async static Task Main(string[] args)
       {
           var filepath = "katiesteve.wav";
           var speechConfig = SpeechConfig.FromSubscription(speechKey, speechRegion);        
           speechConfig.SpeechRecognitionLanguage = "en-US";
           speechConfig.SetProperty(PropertyId.SpeechServiceResponse_DiarizeIntermediateResults, "true"); 
   
           var stopRecognition = new TaskCompletionSource<int>(TaskCreationOptions.RunContinuationsAsynchronously);
   
           // Create an audio stream from a wav file or from the default microphone
           using (var audioConfig = AudioConfig.FromWavFileInput(filepath))
           {
               // Create a conversation transcriber using audio stream input
               using (var conversationTranscriber = new ConversationTranscriber(speechConfig, audioConfig))
               {
                   conversationTranscriber.Transcribing += (s, e) =>
                   {
                       Console.WriteLine($"TRANSCRIBING: Text={e.Result.Text} Speaker ID={e.Result.SpeakerId}");
                   };
   
                   conversationTranscriber.Transcribed += (s, e) =>
                   {
                       if (e.Result.Reason == ResultReason.RecognizedSpeech)
                       {
                           Console.WriteLine();
                           Console.WriteLine($"TRANSCRIBED: Text={e.Result.Text} Speaker ID={e.Result.SpeakerId}");
                           Console.WriteLine();
                       }
                       else if (e.Result.Reason == ResultReason.NoMatch)
                       {
                           Console.WriteLine($"NOMATCH: Speech could not be transcribed.");
                       }
                   };
   
                   conversationTranscriber.Canceled += (s, e) =>
                   {
                       Console.WriteLine($"CANCELED: Reason={e.Reason}");
   
                       if (e.Reason == CancellationReason.Error)
                       {
                           Console.WriteLine($"CANCELED: ErrorCode={e.ErrorCode}");
                           Console.WriteLine($"CANCELED: ErrorDetails={e.ErrorDetails}");
                           Console.WriteLine($"CANCELED: Did you set the speech resource key and region values?");
                           stopRecognition.TrySetResult(0);
                       }
   
                       stopRecognition.TrySetResult(0);
                   };
   
                   conversationTranscriber.SessionStopped += (s, e) =>
                   {
                       Console.WriteLine("\n    Session stopped event.");
                       stopRecognition.TrySetResult(0);
                   };
   
                   await conversationTranscriber.StartTranscribingAsync();
   
                   // Waits for completion. Use Task.WaitAny to keep the task rooted.
                   Task.WaitAny(new[] { stopRecognition.Task });
   
                   await conversationTranscriber.StopTranscribingAsync();
               }
           }
       }
   }
   

4. Get the sample audio file or use your own .wav file. Replace katiesteve.wav with the path and name of your .wav file.

   The application recognizes speech from multiple participants in the conversation. Your audio file should contain multiple speakers.

5. To change the speech recognition language, replace en-US with another supported language. For example, es-ES for Spanish (Spain). The default language is en-US if you don't specify a language. For details about how to identify one of multiple languages that might be spoken, see language identification.

6. Run your console application to start conversation transcription:

   dotnet run
   

Important

Make sure that you set the SPEECH_KEY and SPEECH_REGION environment variables. If you don't set these variables, the sample fails with an error message.

The transcribed conversation should be output as text:

TRANSCRIBING: Text=good morning steve Speaker ID=Unknown
TRANSCRIBING: Text=good morning steve how are Speaker ID=Guest-1
TRANSCRIBING: Text=good morning steve how are you doing today Speaker ID=Guest-1

TRANSCRIBED: Text=Good morning, Steve. How are you doing today? Speaker ID=Guest-1

TRANSCRIBING: Text=good morning katie Speaker ID=Unknown
TRANSCRIBING: Text=good morning katie i hope Speaker ID=Guest-2
TRANSCRIBING: Text=good morning katie i hope you're having a great Speaker ID=Guest-2
TRANSCRIBING: Text=good morning katie i hope you're having a great start to Speaker ID=Guest-2
TRANSCRIBING: Text=good morning katie i hope you're having a great start to your day Speaker ID=Guest-2

TRANSCRIBED: Text=Good morning, Katie. I hope you're having a great start to your day. Speaker ID=Guest-2

TRANSCRIBING: Text=have you tried Speaker ID=Unknown
TRANSCRIBING: Text=have you tried the latest Speaker ID=Unknown
TRANSCRIBING: Text=have you tried the latest real time Speaker ID=Guest-1
TRANSCRIBING: Text=have you tried the latest real time diarization Speaker ID=Guest-1
TRANSCRIBING: Text=have you tried the latest real time diarization in Speaker ID=Guest-1
TRANSCRIBING: Text=have you tried the latest real time diarization in microsoft Speaker ID=Guest-1
TRANSCRIBING: Text=have you tried the latest real time diarization in microsoft speech Speaker ID=Guest-1
TRANSCRIBING: Text=have you tried the latest real time diarization in microsoft speech service Speaker ID=Guest-1
TRANSCRIBING: Text=have you tried the latest real time diarization in microsoft speech service which Speaker ID=Guest-1
TRANSCRIBING: Text=have you tried the latest real time diarization in microsoft speech service which can Speaker ID=Guest-1
TRANSCRIBING: Text=have you tried the latest real time diarization in microsoft speech service which can tell you Speaker ID=Guest-1
TRANSCRIBING: Text=have you tried the latest real time diarization in microsoft speech service which can tell you who said Speaker ID=Guest-1
TRANSCRIBING: Text=have you tried the latest real time diarization in microsoft speech service which can tell you who said what Speaker ID=Guest-1
TRANSCRIBING: Text=have you tried the latest real time diarization in microsoft speech service which can tell you who said what in Speaker ID=Guest-1
TRANSCRIBING: Text=have you tried the latest real time diarization in microsoft speech service which can tell you who said what in real time Speaker ID=Guest-1

TRANSCRIBED: Text=Have you tried the latest real time diarization in Microsoft Speech Service which can tell you who said what in real time? Speaker ID=Guest-1

TRANSCRIBING: Text=not yet Speaker ID=Unknown
TRANSCRIBING: Text=not yet i Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch trans Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization Speaker ID=Guest-2  
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization function Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produc Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces di Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new feature Speaker ID=Guest-2  
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new feature able to Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new feature able to di Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new feature able to diarize Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new feature able to diarize in real Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new feature able to diarize in real time Speaker ID=Guest-2

TRANSCRIBED: Text=Not yet. I've been using the batch transcription with diarization functionality, but it produces diarization results after the whole audio is processed. Is the new feature able to diarize in real time? Speaker ID=Guest-2

TRANSCRIBING: Text=absolutely Speaker ID=Unknown
TRANSCRIBING: Text=absolutely i Speaker ID=Unknown
TRANSCRIBING: Text=absolutely i recom Speaker ID=Guest-1
TRANSCRIBING: Text=absolutely i recommend Speaker ID=Guest-1
TRANSCRIBING: Text=absolutely i recommend you give it a try Speaker ID=Guest-1

TRANSCRIBED: Text=Absolutely, I recommend you give it a try. Speaker ID=Guest-1

TRANSCRIBING: Text=that's exc Speaker ID=Unknown
TRANSCRIBING: Text=that's exciting Speaker ID=Unknown
TRANSCRIBING: Text=that's exciting let me try Speaker ID=Guest-2
TRANSCRIBING: Text=that's exciting let me try it right now Speaker ID=Guest-2

TRANSCRIBED: Text=That's exciting. Let me try it right now. Speaker ID=Guest-2

Speakers are identified as Guest-1, Guest-2, and so on, depending on the number of speakers in the conversation.

Note

You might see Speaker ID=Unknown in some of the early intermediate results when the speaker is not yet identified. Without intermediate diarization results (if you don't set the PropertyId.SpeechServiceResponse_DiarizeIntermediateResults property to "true"), the speaker ID is always "Unknown".

Clean up resources

You can use the Azure portal or Azure Command Line Interface (CLI) to remove the Speech resource you created.

Reference documentation | Package (NuGet) | Additional samples on GitHub

In this quickstart, you run an application for speech to text transcription with real-time diarization. Diarization distinguishes between the different speakers who participate in the conversation. The Speech service provides information about which speaker was speaking a particular part of transcribed speech.

The speaker information is included in the result in the speaker ID field. The speaker ID is a generic identifier assigned to each conversation participant by the service during the recognition as different speakers are being identified from the provided audio content.

Tip

You can try real-time speech to text in Speech Studio without signing up or writing any code. However, the Speech Studio doesn't yet support diarization.

Prerequisites

• An Azure subscription. You can create one for free.
• Create a Speech resource in the Azure portal.
• Get the Speech resource key and region. After your Speech resource is deployed, select Go to resource to view and manage keys.

Set up the environment

The Speech SDK is available as a NuGet package and implements .NET Standard 2.0. You install the Speech SDK later in this guide, but first check the SDK installation guide for any more requirements.

Set environment variables

You need to authenticate your application to access Azure AI services. This article shows you how to use environment variables to store your credentials. You can then access the environment variables from your code to authenticate your application. For production, use a more secure way to store and access your credentials.

Important

We recommend Microsoft Entra ID authentication with managed identities for Azure resources to avoid storing credentials with your applications that run in the cloud.

If you use an API key, store it securely somewhere else, such as in Azure Key Vault. Don't include the API key directly in your code, and never post it publicly.

For more information about AI services security, see Authenticate requests to Azure AI services.

To set the environment variables for your Speech resource key and region, open a console window, and follow the instructions for your operating system and development environment.

• To set the SPEECH_KEY environment variable, replace your-key with one of the keys for your resource.
• To set the SPEECH_REGION environment variable, replace your-region with one of the regions for your resource.

Windows

setx SPEECH_KEY your-key
setx SPEECH_REGION your-region

Note

If you only need to access the environment variables in the current console, you can set the environment variable with set instead of setx.

After you add the environment variables, you might need to restart any programs that need to read the environment variables, including the console window. For example, if you're using Visual Studio as your editor, restart Visual Studio before you run the example.

Linux

Bash

Edit your .bashrc file, and add the environment variables:

export SPEECH_KEY=your-key
export SPEECH_REGION=your-region

After you add the environment variables, run source ~/.bashrc from your console window to make the changes effective.

macOS

Bash

Edit your .bash_profile file, and add the environment variables:

export SPEECH_KEY=your-key
export SPEECH_REGION=your-region

After you add the environment variables, run source ~/.bash_profile from your console window to make the changes effective.

Xcode

For iOS and macOS development, you set the environment variables in Xcode. For example, follow these steps to set the environment variable in Xcode 13.4.1.

1. Select Product > Scheme > Edit scheme.
2. Select Arguments on the Run (Debug Run) page.
3. Under Environment Variables select the plus (+) sign to add a new environment variable.
4. Enter SPEECH_KEY for the Name and enter your Speech resource key for the Value.

To set the environment variable for your Speech resource region, follow the same steps. Set SPEECH_REGION to the region of your resource. For example, westus.

For more configuration options, see the Xcode documentation.

Implement diarization from file with conversation transcription

Follow these steps to create a console application and install the Speech SDK.

1. Create a new C++ console project in Visual Studio Community 2022 named ConversationTranscription.

2. Select Tools > Nuget Package Manager > Package Manager Console. In the Package Manager Console, run this command:

   Install-Package Microsoft.CognitiveServices.Speech
   

3. Replace the contents of ConversationTranscription.cpp with the following code.

   #include <iostream> 
   #include <stdlib.h>
   #include <speechapi_cxx.h>
   #include <future>
   
   using namespace Microsoft::CognitiveServices::Speech;
   using namespace Microsoft::CognitiveServices::Speech::Audio;
   using namespace Microsoft::CognitiveServices::Speech::Transcription;
   
   std::string GetEnvironmentVariable(const char* name);
   
   int main()
   {
       // This example requires environment variables named "SPEECH_KEY" and "SPEECH_REGION"
       auto speechKey = GetEnvironmentVariable("SPEECH_KEY");
       auto speechRegion = GetEnvironmentVariable("SPEECH_REGION");
   
       if ((size(speechKey) == 0) || (size(speechRegion) == 0)) {
           std::cout << "Please set both SPEECH_KEY and SPEECH_REGION environment variables." << std::endl;
           return -1;
       }
   
       auto speechConfig = SpeechConfig::FromSubscription(speechKey, speechRegion);
       speechConfig->SetProperty(PropertyId::SpeechServiceResponse_DiarizeIntermediateResults, "true"); 
   
       speechConfig->SetSpeechRecognitionLanguage("en-US");
   
       auto audioConfig = AudioConfig::FromWavFileInput("katiesteve.wav");
       auto conversationTranscriber = ConversationTranscriber::FromConfig(speechConfig, audioConfig);
   
       // promise for synchronization of recognition end.
       std::promise<void> recognitionEnd;
   
       // Subscribes to events.
       conversationTranscriber->Transcribing.Connect([](const ConversationTranscriptionEventArgs& e)
           {
               std::cout << "TRANSCRIBING:" << e.Result->Text << std::endl;
               std::cout << "Speaker ID=" << e.Result->SpeakerId << std::endl;
           });
   
       conversationTranscriber->Transcribed.Connect([](const ConversationTranscriptionEventArgs& e)
           {
               if (e.Result->Reason == ResultReason::RecognizedSpeech)
               {
                   std::cout << "\n" << "TRANSCRIBED: Text=" << e.Result->Text << std::endl;
                   std::cout << "Speaker ID=" << e.Result->SpeakerId << "\n" << std::endl;
               }
               else if (e.Result->Reason == ResultReason::NoMatch)
               {
                   std::cout << "NOMATCH: Speech could not be transcribed." << std::endl;
               }
           });
   
       conversationTranscriber->Canceled.Connect([&recognitionEnd](const ConversationTranscriptionCanceledEventArgs& e)
           {
               auto cancellation = CancellationDetails::FromResult(e.Result);
               std::cout << "CANCELED: Reason=" << (int)cancellation->Reason << std::endl;
   
               if (cancellation->Reason == CancellationReason::Error)
               {
                   std::cout << "CANCELED: ErrorCode=" << (int)cancellation->ErrorCode << std::endl;
                   std::cout << "CANCELED: ErrorDetails=" << cancellation->ErrorDetails << std::endl;
                   std::cout << "CANCELED: Did you set the speech resource key and region values?" << std::endl;
               }
               else if (cancellation->Reason == CancellationReason::EndOfStream)
               {
                   std::cout << "CANCELED: Reach the end of the file." << std::endl;
               }
           });
   
       conversationTranscriber->SessionStopped.Connect([&recognitionEnd](const SessionEventArgs& e)
           {
               std::cout << "Session stopped.";
               recognitionEnd.set_value(); // Notify to stop recognition.
           });
   
       conversationTranscriber->StartTranscribingAsync().wait();
   
       // Waits for recognition end.
       recognitionEnd.get_future().wait();
   
       conversationTranscriber->StopTranscribingAsync().wait();
   }
   
   std::string GetEnvironmentVariable(const char* name)
   {
   #if defined(_MSC_VER)
       size_t requiredSize = 0;
       (void)getenv_s(&requiredSize, nullptr, 0, name);
       if (requiredSize == 0)
       {
           return "";
       }
       auto buffer = std::make_unique<char[]>(requiredSize);
       (void)getenv_s(&requiredSize, buffer.get(), requiredSize, name);
       return buffer.get();
   #else
       auto value = getenv(name);
       return value ? value : "";
   #endif
   }
   

4. Get the sample audio file or use your own .wav file. Replace katiesteve.wav with the path and name of your .wav file.

   The application recognizes speech from multiple participants in the conversation. Your audio file should contain multiple speakers.

5. To change the speech recognition language, replace en-US with another supported language. For example, es-ES for Spanish (Spain). The default language is en-US if you don't specify a language. For details about how to identify one of multiple languages that might be spoken, see language identification.

6. Build and run your application to start conversation transcription:

   Important

   Make sure that you set the SPEECH_KEY and SPEECH_REGION environment variables. If you don't set these variables, the sample fails with an error message.

The transcribed conversation should be output as text:

TRANSCRIBING:good morning
Speaker ID=Unknown
TRANSCRIBING:good morning steve
Speaker ID=Unknown
TRANSCRIBING:good morning steve how are you doing
Speaker ID=Guest-1
TRANSCRIBING:good morning steve how are you doing today
Speaker ID=Guest-1

TRANSCRIBED: Text=Good morning, Steve. How are you doing today?
Speaker ID=Guest-1

TRANSCRIBING:good
Speaker ID=Unknown
TRANSCRIBING:good morning
Speaker ID=Unknown
TRANSCRIBING:good morning kat
Speaker ID=Unknown
TRANSCRIBING:good morning katie i hope you're having a
Speaker ID=Guest-2
TRANSCRIBING:good morning katie i hope you're having a great start to your day
Speaker ID=Guest-2

TRANSCRIBED: Text=Good morning, Katie. I hope you're having a great start to your day.
Speaker ID=Guest-2

TRANSCRIBING:have you
Speaker ID=Unknown
TRANSCRIBING:have you tried
Speaker ID=Unknown
TRANSCRIBING:have you tried the latest
Speaker ID=Unknown
TRANSCRIBING:have you tried the latest real
Speaker ID=Guest-1
TRANSCRIBING:have you tried the latest real time
Speaker ID=Guest-1
TRANSCRIBING:have you tried the latest real time diarization
Speaker ID=Guest-1
TRANSCRIBING:have you tried the latest real time diarization in
Speaker ID=Guest-1
TRANSCRIBING:have you tried the latest real time diarization in microsoft
Speaker ID=Guest-1
TRANSCRIBING:have you tried the latest real time diarization in microsoft speech
Speaker ID=Guest-1
TRANSCRIBING:have you tried the latest real time diarization in microsoft speech service
Speaker ID=Guest-1
TRANSCRIBING:have you tried the latest real time diarization in microsoft speech service which
Speaker ID=Guest-1
TRANSCRIBING:have you tried the latest real time diarization in microsoft speech service which can
Speaker ID=Guest-1
TRANSCRIBING:have you tried the latest real time diarization in microsoft speech service which can tell you
Speaker ID=Guest-1
TRANSCRIBING:have you tried the latest real time diarization in microsoft speech service which can tell you who said
Speaker ID=Guest-1
TRANSCRIBING:have you tried the latest real time diarization in microsoft speech service which can tell you who said what
Speaker ID=Guest-1
TRANSCRIBING:have you tried the latest real time diarization in microsoft speech service which can tell you who said what in
Speaker ID=Guest-1
TRANSCRIBING:have you tried the latest real time diarization in microsoft speech service which can tell you who said what in real time
Speaker ID=Guest-1

TRANSCRIBED: Text=Have you tried the latest real time diarization in Microsoft Speech Service which can tell you who said what in real time?
Speaker ID=Guest-1

TRANSCRIBING:not yet
Speaker ID=Unknown
TRANSCRIBING:not yet i
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch trans
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch transcription with
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch transcription with diarization
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch transcription with diarization function
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch transcription with diarization functionality
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch transcription with diarization functionality but
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch transcription with diarization functionality but it
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch transcription with diarization functionality but it produces
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch transcription with diarization functionality but it produces di
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch transcription with diarization functionality but it produces diarization
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch transcription with diarization functionality but it produces diarization results
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch transcription with diarization functionality but it produces diarization results after
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new feature
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new feature able to
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new feature able to di
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new feature able to diarize
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new feature able to diarize in real
Speaker ID=Guest-2
TRANSCRIBING:not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new feature able to diarize in real time
Speaker ID=Guest-2

TRANSCRIBED: Text=Not yet. I've been using the batch transcription with diarization functionality, but it produces diarization results after the whole audio is processed. Is the new feature able to diarize in real time?
Speaker ID=Guest-2

TRANSCRIBING:absolutely
Speaker ID=Unknown
TRANSCRIBING:absolutely i
Speaker ID=Unknown
TRANSCRIBING:absolutely i recom
Speaker ID=Guest-1
TRANSCRIBING:absolutely i recommend
Speaker ID=Guest-1
TRANSCRIBING:absolutely i recommend you
Speaker ID=Guest-1
TRANSCRIBING:absolutely i recommend you give it a try
Speaker ID=Guest-1

TRANSCRIBED: Text=Absolutely, I recommend you give it a try.
Speaker ID=Guest-1

TRANSCRIBING:that's exc
Speaker ID=Unknown
TRANSCRIBING:that's exciting
Speaker ID=Unknown
TRANSCRIBING:that's exciting let me
Speaker ID=Guest-2
TRANSCRIBING:that's exciting let me try
Speaker ID=Guest-2
TRANSCRIBING:that's exciting let me try it right now
Speaker ID=Guest-2

TRANSCRIBED: Text=That's exciting. Let me try it right now.
Speaker ID=Guest-2

Speakers are identified as Guest-1, Guest-2, and so on, depending on the number of speakers in the conversation.

Note

You might see Speaker ID=Unknown in some of the early intermediate results when the speaker is not yet identified. Without intermediate diarization results (if you don't set the PropertyId::SpeechServiceResponse_DiarizeIntermediateResults property to "true"), the speaker ID is always "Unknown".

Clean up resources

You can use the Azure portal or Azure Command Line Interface (CLI) to remove the Speech resource you created.

Reference documentation | Package (Go) | Additional samples on GitHub

The Speech SDK for the Go programming language doesn't support conversation transcription. Select another programming language or the Go reference and samples linked from the beginning of this article.

Reference documentation | Additional samples on GitHub

In this quickstart, you run an application for speech to text transcription with real-time diarization. Diarization distinguishes between the different speakers who participate in the conversation. The Speech service provides information about which speaker was speaking a particular part of transcribed speech.

The speaker information is included in the result in the speaker ID field. The speaker ID is a generic identifier assigned to each conversation participant by the service during the recognition as different speakers are being identified from the provided audio content.

Tip

You can try real-time speech to text in Speech Studio without signing up or writing any code. However, the Speech Studio doesn't yet support diarization.

Prerequisites

• An Azure subscription. You can create one for free.
• Create a Speech resource in the Azure portal.
• Get the Speech resource key and region. After your Speech resource is deployed, select Go to resource to view and manage keys.

Set up the environment

To set up your environment, install the Speech SDK. The sample in this quickstart works with the Java Runtime.

1. Install Apache Maven. Then run mvn -v to confirm successful installation.

2. Create a new pom.xml file in the root of your project, and copy the following into it:

   <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
       <modelVersion>4.0.0</modelVersion>
       <groupId>com.microsoft.cognitiveservices.speech.samples</groupId>
       <artifactId>quickstart-eclipse</artifactId>
       <version>1.0.0-SNAPSHOT</version>
       <build>
           <sourceDirectory>src</sourceDirectory>
           <plugins>
           <plugin>
               <artifactId>maven-compiler-plugin</artifactId>
               <version>3.7.0</version>
               <configuration>
               <source>1.8</source>
               <target>1.8</target>
               </configuration>
           </plugin>
           </plugins>
       </build>
       <dependencies>
           <dependency>
           <groupId>com.microsoft.cognitiveservices.speech</groupId>
           <artifactId>client-sdk</artifactId>
           <version>1.42.0</version>
           </dependency>
       </dependencies>
   </project>
   

3. Install the Speech SDK and dependencies.

   mvn clean dependency:copy-dependencies
   

Set environment variables

You need to authenticate your application to access Azure AI services. This article shows you how to use environment variables to store your credentials. You can then access the environment variables from your code to authenticate your application. For production, use a more secure way to store and access your credentials.

Important

We recommend Microsoft Entra ID authentication with managed identities for Azure resources to avoid storing credentials with your applications that run in the cloud.

If you use an API key, store it securely somewhere else, such as in Azure Key Vault. Don't include the API key directly in your code, and never post it publicly.

For more information about AI services security, see Authenticate requests to Azure AI services.

To set the environment variables for your Speech resource key and region, open a console window, and follow the instructions for your operating system and development environment.

• To set the SPEECH_KEY environment variable, replace your-key with one of the keys for your resource.
• To set the SPEECH_REGION environment variable, replace your-region with one of the regions for your resource.

Windows

setx SPEECH_KEY your-key
setx SPEECH_REGION your-region

Note

If you only need to access the environment variables in the current console, you can set the environment variable with set instead of setx.

After you add the environment variables, you might need to restart any programs that need to read the environment variables, including the console window. For example, if you're using Visual Studio as your editor, restart Visual Studio before you run the example.

Linux

Bash

Edit your .bashrc file, and add the environment variables:

export SPEECH_KEY=your-key
export SPEECH_REGION=your-region

After you add the environment variables, run source ~/.bashrc from your console window to make the changes effective.

macOS

Bash

Edit your .bash_profile file, and add the environment variables:

export SPEECH_KEY=your-key
export SPEECH_REGION=your-region

After you add the environment variables, run source ~/.bash_profile from your console window to make the changes effective.

Xcode

For iOS and macOS development, you set the environment variables in Xcode. For example, follow these steps to set the environment variable in Xcode 13.4.1.

1. Select Product > Scheme > Edit scheme.
2. Select Arguments on the Run (Debug Run) page.
3. Under Environment Variables select the plus (+) sign to add a new environment variable.
4. Enter SPEECH_KEY for the Name and enter your Speech resource key for the Value.

To set the environment variable for your Speech resource region, follow the same steps. Set SPEECH_REGION to the region of your resource. For example, westus.

For more configuration options, see the Xcode documentation.

Implement diarization from file with conversation transcription

Follow these steps to create a console application for conversation transcription.

1. Create a new file named ConversationTranscription.java in the same project root directory.

2. Copy the following code into ConversationTranscription.java:

   import com.microsoft.cognitiveservices.speech.*;
   import com.microsoft.cognitiveservices.speech.audio.AudioConfig;
   import com.microsoft.cognitiveservices.speech.transcription.*;
   
   import java.util.concurrent.Semaphore;
   import java.util.concurrent.ExecutionException;
   import java.util.concurrent.Future;
   
   public class ConversationTranscription {
       // This example requires environment variables named "SPEECH_KEY" and "SPEECH_REGION"
       private static String speechKey = System.getenv("SPEECH_KEY");
       private static String speechRegion = System.getenv("SPEECH_REGION");
   
       public static void main(String[] args) throws InterruptedException, ExecutionException {
   
           SpeechConfig speechConfig = SpeechConfig.fromSubscription(speechKey, speechRegion);
           speechConfig.setSpeechRecognitionLanguage("en-US");
           AudioConfig audioInput = AudioConfig.fromWavFileInput("katiesteve.wav");
           speechConfig.setProperty(PropertyId.SpeechServiceResponse_DiarizeIntermediateResults, "true");
   
           Semaphore stopRecognitionSemaphore = new Semaphore(0);
   
           ConversationTranscriber conversationTranscriber = new ConversationTranscriber(speechConfig, audioInput);
           {
               // Subscribes to events.
               conversationTranscriber.transcribing.addEventListener((s, e) -> {
                   System.out.println("TRANSCRIBING: Text=" + e.getResult().getText() + " Speaker ID=" + e.getResult().getSpeakerId() );
               });
   
               conversationTranscriber.transcribed.addEventListener((s, e) -> {
                   if (e.getResult().getReason() == ResultReason.RecognizedSpeech) {
                       System.out.println();
                       System.out.println("TRANSCRIBED: Text=" + e.getResult().getText() + " Speaker ID=" + e.getResult().getSpeakerId() );
                       System.out.println();
                   }
                   else if (e.getResult().getReason() == ResultReason.NoMatch) {
                       System.out.println("NOMATCH: Speech could not be transcribed.");
                   }
               });
   
               conversationTranscriber.canceled.addEventListener((s, e) -> {
                   System.out.println("CANCELED: Reason=" + e.getReason());
   
                   if (e.getReason() == CancellationReason.Error) {
                       System.out.println("CANCELED: ErrorCode=" + e.getErrorCode());
                       System.out.println("CANCELED: ErrorDetails=" + e.getErrorDetails());
                       System.out.println("CANCELED: Did you update the subscription info?");
                   }
   
                   stopRecognitionSemaphore.release();
               });
   
               conversationTranscriber.sessionStarted.addEventListener((s, e) -> {
                   System.out.println("\n    Session started event.");
               });
   
               conversationTranscriber.sessionStopped.addEventListener((s, e) -> {
                   System.out.println("\n    Session stopped event.");
               });
   
               conversationTranscriber.startTranscribingAsync().get();
   
               // Waits for completion.
               stopRecognitionSemaphore.acquire();
   
               conversationTranscriber.stopTranscribingAsync().get();
           }
   
           speechConfig.close();
           audioInput.close();
           conversationTranscriber.close();
   
           System.exit(0);
       }
   }
   

3. Get the sample audio file or use your own .wav file. Replace katiesteve.wav with the path and name of your .wav file.

   The application recognizes speech from multiple participants in the conversation. Your audio file should contain multiple speakers.

4. To change the speech recognition language, replace en-US with another supported language. For example, es-ES for Spanish (Spain). The default language is en-US if you don't specify a language. For details about how to identify one of multiple languages that might be spoken, see language identification.

5. Run your new console application to start conversation transcription:

   javac ConversationTranscription.java -cp ".;target\dependency\*"
   java -cp ".;target\dependency\*" ConversationTranscription
   

Important

Make sure that you set the SPEECH_KEY and SPEECH_REGION environment variables. If you don't set these variables, the sample fails with an error message.

The transcribed conversation should be output as text:

TRANSCRIBING: Text=good morning Speaker ID=Unknown
TRANSCRIBING: Text=good morning steve Speaker ID=Unknown
TRANSCRIBING: Text=good morning steve how Speaker ID=Guest-1
TRANSCRIBING: Text=good morning steve how are you doing Speaker ID=Guest-1
TRANSCRIBING: Text=good morning steve how are you doing today Speaker ID=Guest-1

TRANSCRIBED: Text=Good morning, Steve. How are you doing today? Speaker ID=Guest-1

TRANSCRIBING: Text=good morning katie i hope Speaker ID=Guest-2
TRANSCRIBING: Text=good morning katie i hope you're having a Speaker ID=Guest-2
TRANSCRIBING: Text=good morning katie i hope you're having a great Speaker ID=Guest-2
TRANSCRIBING: Text=good morning katie i hope you're having a great start to Speaker ID=Guest-2
TRANSCRIBING: Text=good morning katie i hope you're having a great start to your day Speaker ID=Guest-2

TRANSCRIBED: Text=Good morning, Katie. I hope you're having a great start to your day. Speaker ID=Guest-2

TRANSCRIBING: Text=have you tried Speaker ID=Unknown
TRANSCRIBING: Text=have you tried the latest Speaker ID=Unknown
TRANSCRIBING: Text=have you tried the latest real Speaker ID=Guest-1
TRANSCRIBING: Text=have you tried the latest real time Speaker ID=Guest-1
TRANSCRIBING: Text=have you tried the latest real time diarization Speaker ID=Guest-1
TRANSCRIBING: Text=have you tried the latest real time diarization in Speaker ID=Guest-1
TRANSCRIBING: Text=have you tried the latest real time diarization in microsoft Speaker ID=Guest-1
TRANSCRIBING: Text=have you tried the latest real time diarization in microsoft speech Speaker ID=Guest-1
TRANSCRIBING: Text=have you tried the latest real time diarization in microsoft speech service Speaker ID=Guest-1
TRANSCRIBING: Text=have you tried the latest real time diarization in microsoft speech service which Speaker ID=Guest-1
TRANSCRIBING: Text=have you tried the latest real time diarization in microsoft speech service which can Speaker ID=Guest-1
TRANSCRIBING: Text=have you tried the latest real time diarization in microsoft speech service which can tell you Speaker ID=Guest-1
TRANSCRIBING: Text=have you tried the latest real time diarization in microsoft speech service which can tell you who said Speaker ID=Guest-1
TRANSCRIBING: Text=have you tried the latest real time diarization in microsoft speech service which can tell you who said what Speaker ID=Guest-1
TRANSCRIBING: Text=have you tried the latest real time diarization in microsoft speech service which can tell you who said what in Speaker ID=Guest-1
TRANSCRIBING: Text=have you tried the latest real time diarization in microsoft speech service which can tell you who said what in real time Speaker ID=Guest-1

TRANSCRIBED: Text=Have you tried the latest real time diarization in Microsoft Speech Service which can tell you who said what in real time? Speaker ID=Guest-1

TRANSCRIBING: Text=not yet Speaker ID=Unknown
TRANSCRIBING: Text=not yet i Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch trans Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization function Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces di Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new feature Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new feature able to Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new feature able to di Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new feature able to diarize in real Speaker ID=Guest-2
TRANSCRIBING: Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new feature able to diarize in real time Speaker ID=Guest-2

TRANSCRIBED: Text=Not yet. I've been using the batch transcription with diarization functionality, but it produces diarization results after the whole audio is processed. Is the new feature able to diarize in real time? Speaker ID=Guest-2

TRANSCRIBING: Text=absolutely Speaker ID=Unknown
TRANSCRIBING: Text=absolutely i recom Speaker ID=Guest-1
TRANSCRIBING: Text=absolutely i recommend Speaker ID=Guest-1
TRANSCRIBING: Text=absolutely i recommend you Speaker ID=Guest-1
TRANSCRIBING: Text=absolutely i recommend you give it a try Speaker ID=Guest-1

TRANSCRIBED: Text=Absolutely, I recommend you give it a try. Speaker ID=Guest-1

TRANSCRIBING: Text=that's exc Speaker ID=Unknown
TRANSCRIBING: Text=that's exciting Speaker ID=Unknown
TRANSCRIBING: Text=that's exciting let me try Speaker ID=Guest-2
TRANSCRIBING: Text=that's exciting let me try it right now Speaker ID=Guest-2

TRANSCRIBED: Text=That's exciting. Let me try it right now. Speaker ID=Guest-2

Speakers are identified as Guest-1, Guest-2, and so on, depending on the number of speakers in the conversation.

Note

You might see Speaker ID=Unknown in some of the early intermediate results when the speaker is not yet identified. Without intermediate diarization results (if you don't set the PropertyId.SpeechServiceResponse_DiarizeIntermediateResults property to "true"), the speaker ID is always "Unknown".

Clean up resources

You can use the Azure portal or Azure Command Line Interface (CLI) to remove the Speech resource you created.

Reference documentation | Package (npm) | Additional samples on GitHub | Library source code

In this quickstart, you run an application for speech to text transcription with real-time diarization. Diarization distinguishes between the different speakers who participate in the conversation. The Speech service provides information about which speaker was speaking a particular part of transcribed speech.

The speaker information is included in the result in the speaker ID field. The speaker ID is a generic identifier assigned to each conversation participant by the service during the recognition as different speakers are being identified from the provided audio content.

Tip

You can try real-time speech to text in Speech Studio without signing up or writing any code. However, the Speech Studio doesn't yet support diarization.

Prerequisites

• An Azure subscription. You can create one for free.
• Create a Speech resource in the Azure portal.
• Get the Speech resource key and region. After your Speech resource is deployed, select Go to resource to view and manage keys.

Set up the environment

To set up your environment, install the Speech SDK for JavaScript. If you just want the package name to install, run npm install microsoft-cognitiveservices-speech-sdk. For guided installation instructions, see the SDK installation guide.

Set environment variables

You need to authenticate your application to access Azure AI services. This article shows you how to use environment variables to store your credentials. You can then access the environment variables from your code to authenticate your application. For production, use a more secure way to store and access your credentials.

Important

We recommend Microsoft Entra ID authentication with managed identities for Azure resources to avoid storing credentials with your applications that run in the cloud.

If you use an API key, store it securely somewhere else, such as in Azure Key Vault. Don't include the API key directly in your code, and never post it publicly.

For more information about AI services security, see Authenticate requests to Azure AI services.

To set the environment variables for your Speech resource key and region, open a console window, and follow the instructions for your operating system and development environment.

• To set the SPEECH_KEY environment variable, replace your-key with one of the keys for your resource.
• To set the SPEECH_REGION environment variable, replace your-region with one of the regions for your resource.

Windows

setx SPEECH_KEY your-key
setx SPEECH_REGION your-region

Note

If you only need to access the environment variables in the current console, you can set the environment variable with set instead of setx.

After you add the environment variables, you might need to restart any programs that need to read the environment variables, including the console window. For example, if you're using Visual Studio as your editor, restart Visual Studio before you run the example.

Linux

Bash

Edit your .bashrc file, and add the environment variables:

export SPEECH_KEY=your-key
export SPEECH_REGION=your-region

After you add the environment variables, run source ~/.bashrc from your console window to make the changes effective.

macOS

Bash

Edit your .bash_profile file, and add the environment variables:

export SPEECH_KEY=your-key
export SPEECH_REGION=your-region

After you add the environment variables, run source ~/.bash_profile from your console window to make the changes effective.

Xcode

For iOS and macOS development, you set the environment variables in Xcode. For example, follow these steps to set the environment variable in Xcode 13.4.1.

1. Select Product > Scheme > Edit scheme.
2. Select Arguments on the Run (Debug Run) page.
3. Under Environment Variables select the plus (+) sign to add a new environment variable.
4. Enter SPEECH_KEY for the Name and enter your Speech resource key for the Value.

To set the environment variable for your Speech resource region, follow the same steps. Set SPEECH_REGION to the region of your resource. For example, westus.

For more configuration options, see the Xcode documentation.

Implement diarization from file with conversation transcription

Follow these steps to create a new console application for conversation transcription.

1. Open a command prompt window where you want the new project, and create a new file named ConversationTranscription.js.

2. Install the Speech SDK for JavaScript:

   npm install microsoft-cognitiveservices-speech-sdk
   

3. Copy the following code into ConversationTranscription.js:

   const fs = require("fs");
   const sdk = require("microsoft-cognitiveservices-speech-sdk");
   
   // This example requires environment variables named "SPEECH_KEY" and "SPEECH_REGION"
   const speechConfig = sdk.SpeechConfig.fromSubscription(process.env.SPEECH_KEY, process.env.SPEECH_REGION);
   
   function fromFile() {
       const filename = "katiesteve.wav";
   
       let audioConfig = sdk.AudioConfig.fromWavFileInput(fs.readFileSync(filename));
       let conversationTranscriber = new sdk.ConversationTranscriber(speechConfig, audioConfig);
   
       var pushStream = sdk.AudioInputStream.createPushStream();
   
       fs.createReadStream(filename).on('data', function(arrayBuffer) {
           pushStream.write(arrayBuffer.slice());
       }).on('end', function() {
           pushStream.close();
       });
   
       console.log("Transcribing from: " + filename);
   
       conversationTranscriber.sessionStarted = function(s, e) {
           console.log("SessionStarted event");
           console.log("SessionId:" + e.sessionId);
       };
       conversationTranscriber.sessionStopped = function(s, e) {
           console.log("SessionStopped event");
           console.log("SessionId:" + e.sessionId);
           conversationTranscriber.stopTranscribingAsync();
       };
       conversationTranscriber.canceled = function(s, e) {
           console.log("Canceled event");
           console.log(e.errorDetails);
           conversationTranscriber.stopTranscribingAsync();
       };
       conversationTranscriber.transcribed = function(s, e) {
           console.log("TRANSCRIBED: Text=" + e.result.text + " Speaker ID=" + e.result.speakerId);
       };
   
       // Start conversation transcription
       conversationTranscriber.startTranscribingAsync(
           function () {},
           function (err) {
               console.trace("err - starting transcription: " + err);
           }
       );
   
   }
   fromFile();
   

4. Get the sample audio file or use your own .wav file. Replace katiesteve.wav with the path and name of your .wav file.

   The application recognizes speech from multiple participants in the conversation. Your audio file should contain multiple speakers.

5. To change the speech recognition language, replace en-US with another supported language. For example, es-ES for Spanish (Spain). The default language is en-US if you don't specify a language. For details about how to identify one of multiple languages that might be spoken, see language identification.

6. Run your new console application to start speech recognition from a file:

   node.exe ConversationTranscription.js
   

Important

Make sure that you set the SPEECH_KEY and SPEECH_REGION environment variables. If you don't set these variables, the sample fails with an error message.

The transcribed conversation should be output as text:

SessionStarted event
SessionId:E87AFBA483C2481985F6C9AF719F616B
TRANSCRIBED: Text=Good morning, Steve. Speaker ID=Unknown
TRANSCRIBED: Text=Good morning, Katie. Speaker ID=Unknown
TRANSCRIBED: Text=Have you tried the latest real time diarization in Microsoft Speech Service which can tell you who said what in real time? Speaker ID=Guest-1
TRANSCRIBED: Text=Not yet. I've been using the batch transcription with diarization functionality, but it produces diarization result until whole audio get processed. Speaker ID=Guest-2
TRANSCRIBED: Text=Is the new feature can diarize in real time? Speaker ID=Guest-2
TRANSCRIBED: Text=Absolutely. Speaker ID=Guest-1
TRANSCRIBED: Text=That's exciting. Let me try it right now. Speaker ID=Guest-2
Canceled event
undefined
SessionStopped event
SessionId:E87AFBA483C2481985F6C9AF719F616B

Speakers are identified as Guest-1, Guest-2, and so on, depending on the number of speakers in the conversation.

Clean up resources

You can use the Azure portal or Azure Command Line Interface (CLI) to remove the Speech resource you created.

Reference documentation | Package (download) | Additional samples on GitHub

The Speech SDK for Objective-C does support conversation transcription, but we haven't yet included a guide here. Please select another programming language to get started and learn about the concepts, or see the Objective-C reference and samples linked from the beginning of this article.

Reference documentation | Package (download) | Additional samples on GitHub

The Speech SDK for Swift does support conversation transcription, but we haven't yet included a guide here. Please select another programming language to get started and learn about the concepts, or see the Swift reference and samples linked from the beginning of this article.

Reference documentation | Package (PyPi) | Additional samples on GitHub

In this quickstart, you run an application for speech to text transcription with real-time diarization. Diarization distinguishes between the different speakers who participate in the conversation. The Speech service provides information about which speaker was speaking a particular part of transcribed speech.

The speaker information is included in the result in the speaker ID field. The speaker ID is a generic identifier assigned to each conversation participant by the service during the recognition as different speakers are being identified from the provided audio content.

Tip

You can try real-time speech to text in Speech Studio without signing up or writing any code. However, the Speech Studio doesn't yet support diarization.

Prerequisites

• An Azure subscription. You can create one for free.
• Create a Speech resource in the Azure portal.
• Get the Speech resource key and region. After your Speech resource is deployed, select Go to resource to view and manage keys.

Set up the environment

The Speech SDK for Python is available as a Python Package Index (PyPI) module. The Speech SDK for Python is compatible with Windows, Linux, and macOS.

• You must install the Microsoft Visual C++ Redistributable for Visual Studio 2015, 2017, 2019, and 2022 for your platform. Installing this package for the first time might require a restart.
• On Linux, you must use the x64 target architecture.

Install a version of Python from 3.7 or later. First check the SDK installation guide for any more requirements.

Set environment variables

You need to authenticate your application to access Azure AI services. This article shows you how to use environment variables to store your credentials. You can then access the environment variables from your code to authenticate your application. For production, use a more secure way to store and access your credentials.

Important

We recommend Microsoft Entra ID authentication with managed identities for Azure resources to avoid storing credentials with your applications that run in the cloud.

If you use an API key, store it securely somewhere else, such as in Azure Key Vault. Don't include the API key directly in your code, and never post it publicly.

For more information about AI services security, see Authenticate requests to Azure AI services.

To set the environment variables for your Speech resource key and region, open a console window, and follow the instructions for your operating system and development environment.

• To set the SPEECH_KEY environment variable, replace your-key with one of the keys for your resource.
• To set the SPEECH_REGION environment variable, replace your-region with one of the regions for your resource.

Windows

setx SPEECH_KEY your-key
setx SPEECH_REGION your-region

Note

If you only need to access the environment variables in the current console, you can set the environment variable with set instead of setx.

After you add the environment variables, you might need to restart any programs that need to read the environment variables, including the console window. For example, if you're using Visual Studio as your editor, restart Visual Studio before you run the example.

Linux

Bash

Edit your .bashrc file, and add the environment variables:

export SPEECH_KEY=your-key
export SPEECH_REGION=your-region

After you add the environment variables, run source ~/.bashrc from your console window to make the changes effective.

macOS

Bash

Edit your .bash_profile file, and add the environment variables:

export SPEECH_KEY=your-key
export SPEECH_REGION=your-region

After you add the environment variables, run source ~/.bash_profile from your console window to make the changes effective.

Xcode

For iOS and macOS development, you set the environment variables in Xcode. For example, follow these steps to set the environment variable in Xcode 13.4.1.

1. Select Product > Scheme > Edit scheme.
2. Select Arguments on the Run (Debug Run) page.
3. Under Environment Variables select the plus (+) sign to add a new environment variable.
4. Enter SPEECH_KEY for the Name and enter your Speech resource key for the Value.

To set the environment variable for your Speech resource region, follow the same steps. Set SPEECH_REGION to the region of your resource. For example, westus.

For more configuration options, see the Xcode documentation.

Implement diarization from file with conversation transcription

Follow these steps to create a new console application.

1. Open a command prompt window where you want the new project, and create a new file named conversation_transcription.py.

2. Run this command to install the Speech SDK:

   pip install azure-cognitiveservices-speech
   

3. Copy the following code into conversation_transcription.py:

   import os
   import time
   import azure.cognitiveservices.speech as speechsdk
   
   def conversation_transcriber_recognition_canceled_cb(evt: speechsdk.SessionEventArgs):
       print('Canceled event')
   
   def conversation_transcriber_session_stopped_cb(evt: speechsdk.SessionEventArgs):
       print('SessionStopped event')
   
   def conversation_transcriber_transcribed_cb(evt: speechsdk.SpeechRecognitionEventArgs):
       print('\nTRANSCRIBED:')
       if evt.result.reason == speechsdk.ResultReason.RecognizedSpeech:
           print('\tText={}'.format(evt.result.text))
           print('\tSpeaker ID={}\n'.format(evt.result.speaker_id))
       elif evt.result.reason == speechsdk.ResultReason.NoMatch:
           print('\tNOMATCH: Speech could not be TRANSCRIBED: {}'.format(evt.result.no_match_details))
   
   def conversation_transcriber_transcribing_cb(evt: speechsdk.SpeechRecognitionEventArgs):
       print('TRANSCRIBING:')
       print('\tText={}'.format(evt.result.text))
       print('\tSpeaker ID={}'.format(evt.result.speaker_id))
   
   def conversation_transcriber_session_started_cb(evt: speechsdk.SessionEventArgs):
       print('SessionStarted event')
   
   def recognize_from_file():
       # This example requires environment variables named "SPEECH_KEY" and "SPEECH_REGION"
       speech_config = speechsdk.SpeechConfig(subscription=os.environ.get('SPEECH_KEY'), region=os.environ.get('SPEECH_REGION'))
       speech_config.speech_recognition_language="en-US"
       speech_config.set_property(property_id=speechsdk.PropertyId.SpeechServiceResponse_DiarizeIntermediateResults, value='true')
   
       audio_config = speechsdk.audio.AudioConfig(filename="katiesteve.wav")
       conversation_transcriber = speechsdk.transcription.ConversationTranscriber(speech_config=speech_config, audio_config=audio_config)
   
       transcribing_stop = False
   
       def stop_cb(evt: speechsdk.SessionEventArgs):
           #"""callback that signals to stop continuous recognition upon receiving an event `evt`"""
           print('CLOSING on {}'.format(evt))
           nonlocal transcribing_stop
           transcribing_stop = True
   
       # Connect callbacks to the events fired by the conversation transcriber
       conversation_transcriber.transcribed.connect(conversation_transcriber_transcribed_cb)
       conversation_transcriber.transcribing.connect(conversation_transcriber_transcribing_cb)
       conversation_transcriber.session_started.connect(conversation_transcriber_session_started_cb)
       conversation_transcriber.session_stopped.connect(conversation_transcriber_session_stopped_cb)
       conversation_transcriber.canceled.connect(conversation_transcriber_recognition_canceled_cb)
       # stop transcribing on either session stopped or canceled events
       conversation_transcriber.session_stopped.connect(stop_cb)
       conversation_transcriber.canceled.connect(stop_cb)
   
       conversation_transcriber.start_transcribing_async()
   
       # Waits for completion.
       while not transcribing_stop:
           time.sleep(.5)
   
       conversation_transcriber.stop_transcribing_async()
   
   # Main
   
   try:
       recognize_from_file()
   except Exception as err:
       print("Encountered exception. {}".format(err))
   

4. Get the sample audio file or use your own .wav file. Replace katiesteve.wav with the path and name of your .wav file.

   The application recognizes speech from multiple participants in the conversation. Your audio file should contain multiple speakers.

5. To change the speech recognition language, replace en-US with another supported language. For example, es-ES for Spanish (Spain). The default language is en-US if you don't specify a language. For details about how to identify one of multiple languages that might be spoken, see language identification.

6. Run your new console application to start conversation transcription:

   python conversation_transcription.py
   

Important

Make sure that you set the SPEECH_KEY and SPEECH_REGION environment variables. If you don't set these variables, the sample fails with an error message.

The transcribed conversation should be output as text:

TRANSCRIBING:
        Text=good morning
        Speaker ID=Unknown
TRANSCRIBING:
        Text=good morning steve
        Speaker ID=Unknown
TRANSCRIBING:
        Text=good morning steve how are
        Speaker ID=Guest-1
TRANSCRIBING:
        Text=good morning steve how are you doing today
        Speaker ID=Guest-1

TRANSCRIBED:
        Text=Good morning, Steve. How are you doing today?
        Speaker ID=Guest-1

TRANSCRIBING:
        Text=good morning katie
        Speaker ID=Unknown
TRANSCRIBING:
        Text=good morning katie i hope you're having a
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=good morning katie i hope you're having a great start to
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=good morning katie i hope you're having a great start to your day
        Speaker ID=Guest-2

TRANSCRIBED:
        Text=Good morning, Katie. I hope you're having a great start to your day.
        Speaker ID=Guest-2

TRANSCRIBING:
        Text=have you
        Speaker ID=Unknown
TRANSCRIBING:
        Text=have you tried
        Speaker ID=Unknown
TRANSCRIBING:
        Text=have you tried the latest
        Speaker ID=Unknown
TRANSCRIBING:
        Text=have you tried the latest real
        Speaker ID=Guest-1
TRANSCRIBING:
        Text=have you tried the latest real time
        Speaker ID=Guest-1
TRANSCRIBING:
        Text=have you tried the latest real time diarization
        Speaker ID=Guest-1
TRANSCRIBING:
        Text=have you tried the latest real time diarization in
        Speaker ID=Guest-1
TRANSCRIBING:
        Text=have you tried the latest real time diarization in microsoft
        Speaker ID=Guest-1
TRANSCRIBING:
        Text=have you tried the latest real time diarization in microsoft speech
        Speaker ID=Guest-1
TRANSCRIBING:
        Text=have you tried the latest real time diarization in microsoft speech service
        Speaker ID=Guest-1
TRANSCRIBING:
        Text=have you tried the latest real time diarization in microsoft speech service which
        Speaker ID=Guest-1
TRANSCRIBING:
        Text=have you tried the latest real time diarization in microsoft speech service which can
        Speaker ID=Guest-1
TRANSCRIBING:
        Text=have you tried the latest real time diarization in microsoft speech service which can tell you
        Speaker ID=Guest-1
TRANSCRIBING:
        Text=have you tried the latest real time diarization in microsoft speech service which can tell you who said        
        Speaker ID=Guest-1
TRANSCRIBING:
        Text=have you tried the latest real time diarization in microsoft speech service which can tell you who said what   
        Speaker ID=Guest-1
TRANSCRIBING:
        Text=have you tried the latest real time diarization in microsoft speech service which can tell you who said what in
        Speaker ID=Guest-1
TRANSCRIBING:
        Text=have you tried the latest real time diarization in microsoft speech service which can tell you who said what in real time
        Speaker ID=Guest-1

TRANSCRIBED:
        Text=Have you tried the latest real time diarization in Microsoft Speech Service which can tell you who said what in real time?
        Speaker ID=Guest-1

TRANSCRIBING:
        Text=not yet
        Speaker ID=Unknown
TRANSCRIBING:
        Text=not yet i
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the batch
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the batch trans
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the batch transcription with
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the batch transcription with diarization
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the batch transcription with diarization function
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the batch transcription with diarization functionality
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the batch transcription with diarization functionality but
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the batch transcription with diarization functionality but it
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the batch transcription with diarization functionality but it produces
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the batch transcription with diarization functionality but it produces di
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization     
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new feature
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new feature able to
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new feature able to di
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new feature able to diarize in real
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=not yet i've been using the batch transcription with diarization functionality but it produces diarization results after the whole audio is processed is the new feature able to diarize in real time
        Speaker ID=Guest-2

TRANSCRIBED:
        Text=Not yet. I've been using the batch transcription with diarization functionality, but it produces diarization results after the whole audio is processed. Is the new feature able to diarize in real time?
        Speaker ID=Guest-2

TRANSCRIBING:
        Text=absolutely
        Speaker ID=Unknown
TRANSCRIBING:
        Text=absolutely i
        Speaker ID=Unknown
TRANSCRIBING:
        Text=absolutely i recom
        Speaker ID=Guest-1
TRANSCRIBING:
        Text=absolutely i recommend
        Speaker ID=Guest-1
TRANSCRIBING:
        Text=absolutely i recommend you give it a try
        Speaker ID=Guest-1

TRANSCRIBED:
        Text=Absolutely, I recommend you give it a try.
        Speaker ID=Guest-1

TRANSCRIBING:
        Text=that's exc
        Speaker ID=Unknown
TRANSCRIBING:
        Text=that's exciting
        Speaker ID=Unknown
TRANSCRIBING:
        Text=that's exciting let me
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=that's exciting let me try
        Speaker ID=Guest-2
TRANSCRIBING:
        Text=that's exciting let me try it right now
        Speaker ID=Guest-2

TRANSCRIBED:
        Text=That's exciting. Let me try it right now.
        Speaker ID=Guest-2

Speakers are identified as Guest-1, Guest-2, and so on, depending on the number of speakers in the conversation.

Note

You might see Speaker ID=Unknown in some of the early intermediate results when the speaker is not yet identified. Without intermediate diarization results (if you don't set the PropertyId.SpeechServiceResponse_DiarizeIntermediateResults property to "true"), the speaker ID is always "Unknown".

Clean up resources

You can use the Azure portal or Azure Command Line Interface (CLI) to remove the Speech resource you created.

Speech to text REST API reference | Speech to text REST API for short audio reference | Additional samples on GitHub

The REST API doesn't support conversation transcription. Please select another programming language or tool from the top of this page.

The Speech CLI doesn't support conversation transcription. Please select another programming language or tool from the top of this page.

Next step

Learn more about speech recognition