Cannot Connect Locally to when using Microsoft.CognitiveServices.Speech

Michael Lanctot 0

Hello Everyone,

I am trying to build a method that detects the language of the audio I provide it. My application translates audio but since we batch transcribe and also use a custom model, I cannot use languageIdentification. Therefore, I am trying to detect the language of the audio prior to sending it for transcription so I can give the locale dynamically.

I installed Microsoft.CognitiveServices.Speech via Microsoft Visual Studio (most recent stable build). We utilize Azure Blob services for this project and pull in wav files and convert them to memory streams in the app. I am trying to take these memory streams of audio and pass it to the SDK to detect its language. Here is my method:

    public async Task

navba-MSFT 27,100 Reputation points Microsoft Employee

2025-01-06T04:43:33.5733333+00:00
@Michael Lanctot Welcome to Microsoft Q&A Forum, Thank you for posting your query here!

.

Please confirm if you are using AutoDetectSourceLanguageConfig.FromLanguages(String[]) Method to specify the possible source languages. This helps the SDK to identify the language from the provided audio.

Did you use the RecognizeOnceAsync method to detect the language. More info here.

var result = await recognizer.RecognizeOnceAsync().ConfigureAwait(false); if (result.Reason == ResultReason.TranslatedSpeech) { var lidResult = result.Properties.GetProperty(PropertyId.SpeechServiceConnection_AutoDetectSourceLanguageResult); Console.WriteLine($"RECOGNIZED in '{lidResult}': Text={result.Text}");

For some reason your code is not shared in the issue description. Please try sharing it again.

.

Awaiting your reply.

Michael Lanctot 0

My apologies I didn't realize the code did not completely load in. Here is our code:

public async Task<string> DetectLanguageOfAudio(Stream? audioStream, CancellationToken cancellationToken)
{
    if (audioStream == null)
    {
        return string.Empty;
    }

    if (audioStream.Length == 0)
    {
        throw new InvalidOperationException("The audio stream is empty.");
    }

    var endpointUrl = new Uri(config.AzureSpeechService!.DetectLanguageEndpoint!);
    var detectConfig = SpeechConfig.FromEndpoint(endpointUrl, config.AzureSpeechService!.Key!);

    detectConfig.SetProperty(PropertyId.SpeechServiceConnection_LanguageIdMode, "Continuous");
    
    var logFilePath = @"C:\Users\mlanctot\OneDrive - United Wholesale Mortgage\Documents\Logs\speech_sdk_log.txt";
    detectConfig.SetProperty(PropertyId.Speech_LogFilename, logFilePath);

    var autoDetectSourceLanguageConfig = AutoDetectSourceLanguageConfig.FromLanguages(expectedLanguages);

    var pushStream = AudioInputStream.CreatePushStream();
    var stopRecognition = new TaskCompletionSource<string>();

    _ = Task.Run(async () =>
    {
        byte[] buffer = new byte[4096];
        int bytesRead;

        while ((bytesRead = await audioStream.ReadAsync(buffer)) > 0)
        {
            pushStream.Write(buffer, bytesRead);
        }

        pushStream.Close();
    }, cancellationToken);

    using var audioInput = AudioConfig.FromStreamInput(pushStream);
    using var recognizer = new SpeechRecognizer(detectConfig, autoDetectSourceLanguageConfig, audioInput);

    Console.WriteLine("Attempting to detect langauge");

    string detectedLanguage = string.Empty;

    recognizer.Recognizing += (s, e) =>
    {
        if (e.Result.Reason == ResultReason.RecognizingSpeech)
        {
            var autoDetectSourceLanguageResult = AutoDetectSourceLanguageResult.FromResult(e.Result);
            detectedLanguage = autoDetectSourceLanguageResult.Language; // Capture the detected language
        }
    };

    recognizer.Recognized += (s, e) =>
    {
        if (e.Result.Reason == ResultReason.RecognizedSpeech)
        {
            var autoDetectSourceLanguageResult = AutoDetectSourceLanguageResult.FromResult(e.Result);
            detectedLanguage = autoDetectSourceLanguageResult.Language; // Capture the detected language
        }
    };

    recognizer.Canceled += (s, e) =>
    {
        Console.WriteLine($"CANCELED: Reason={e.Reason}");

        if (e.Reason == CancellationReason.Error)
        {
            Console.WriteLine($"CANCELED: ErrorCode={e.ErrorCode}");
            Console.WriteLine($"CANCELED: ErrorDetails={e.ErrorDetails}");
        }

        stopRecognition.TrySetResult(detectedLanguage); // Return the detected language
    };

    recognizer.SessionStarted += (s, e) =>
    {
        Console.WriteLine("\nSession started event.");
    };

    recognizer.SessionStopped += (s, e) =>
    {
        Console.WriteLine("\nSession stopped event.");
        stopRecognition.TrySetResult(detectedLanguage); // Return the detected language
    };

    // Start continuous recognition
    await recognizer.StartContinuousRecognitionAsync().ConfigureAwait(false);

    // Wait until recognition is stopped
    await stopRecognition.Task;

    // Stop recognition
    await recognizer.StopContinuousRecognitionAsync().ConfigureAwait(false);

    audioStream.Position = 0;

    return detectedLanguage; // Return the detected language
}

Michael Lanctot 0 Reputation points

2025-01-06T21:02:45.2733333+00:00

Also found the log file for this as well.Cspeech_sdk_log.txt

navba-MSFT 27,100 Microsoft Employee

@Michael Lanctot Thanks for sharing the details. Within the Speech SDK logs, I could see the below:


[606712]: 12842ms SPX_TRACE_ERROR: AZ_LOG_ERROR:  tlsio_openssl.c:2464 FORCE-Closing tlsio instance.

[606712]: 12844ms SPX_TRACE_SCOPE_ENTER:  uws_web_socket.cpp:247 OnWebSocketOpened

[606712]: 12845ms SPX_TRACE_ERROR:  web_socket.cpp:902 WS open operation failed with result=1(WS_OPEN_ERROR_UNDERLYING_IO_OPEN_FAILED), code=2573[0x00000a0d], time=2025-01-06T14:19:44.6496149Z

[606712]: 12845ms SPX_TRACE_INFO:  usp_connection.cpp:932 TS:270, TransportError: connection:0xa401b950, code=5, string=Connection failed (no connection to the remote host). Internal error: 1. Error details: Failed with error: WS_OPEN_ERROR_UNDERLYING_IO_OPEN_FAILED

wss://eastus2.stt.speech.microsoft.com/speech/universal/v2

X-ConnectionId: 09ce9fa5e8c14d81900fae21a6065cd1

[606712]: 12846ms SPX_TRACE_ERROR:  usp_reco_engine_adapter.cpp:1997 Response: On Error: Code:5, Message: Connection failed (no connection to the remote host). Internal error: 1. Error details: Failed with error: WS_OPEN_ERROR_UNDERLYING_IO_OPEN_FAILED

wss://eastus2.stt.speech.microsoft.com/speech/universal/v2

X-ConnectionId: 09ce9fa5e8c14d81900fae21a6065cd1.

[606712]: 12847ms SPX_DBG_TRACE_VERBOSE:  usp_reco_engine_adapter.cpp:3066 TryChangeState; audioState/uspState: 2/1000 => 2/-1 USP-ERRORERROR

[606712]: 12847ms SPX_TRACE_ERROR:  usp_reco_engine_adapter.cpp:2011 OnError: site->Error() ... error='Connection failed (no connection to the remote host). Internal error: 1. Error details: Failed with error: WS_OPEN_ERROR_UNDERLYING_IO_OPEN_FAILED

wss://eastus2.stt.speech.microsoft.com/speech/universal/v2

X-ConnectionId: 09ce9fa5e8c14d81900fae21a6065cd1'

In Azure Portal for your Azure Speech resource, Could you please check if your Azure Speech resource --> **Networking **tab has any firewall / VNET configured ?
Within your Corporate network, do you have any proxy / firewall ?
Can you try using ipv4 network settings instead of ipv6 ?

Share via

Cannot Connect Locally to when using Microsoft.CognitiveServices.Speech

Your answer