Azure Cognitive Speech Services Failed with error: WS_OPEN_ERROR_UNDERLYING_IO_OPEN_FAILED

Question

I'm having an issue to connect to the Azure Speech Services custom domain using Entra Id. When I create a SpeechConfig using Domain and Subscription key, everything is working fine, but when I try to use Aad token, the the connection fails.

This is my code:

var resourceId = _options.Value.ResourceId;
string aadToken = await GetAadToken();
var region = _options.Value.Region;
// You need to include the "aad#" prefix and the "#" (hash) separator between resource ID and AAD access token.
var authorizationToken = $"aad#{resourceId}#{aadToken}";
var config = SpeechConfig.FromAuthorizationToken(authorizationToken, region);
config.SpeechRecognitionLanguage = language;
return config;

The speech recognizer ends with this error:

Speech recognizing canceled with Connection failed (no connection to the remote host). Internal error: 1. Error details: Failed with error: WS_OPEN_ERROR_UNDERLYING_IO_OPEN_FAILED

wss://westeurope.stt.speech.microsoft.com/speech/universal/v2

X-ConnectionId: d29f5c80cbc04ee5b90ca20e73740a0a SessionId: d29f5c80cbc04ee5b90ca20e73740a0a

My working example is as follows:

var domain = new Uri(_options.Value.Endpoint).DnsSafeHost;
var uri = new Uri($"wss://{domain}/stt/speech/recognition/conversation/cognitiveservices/v1");
var config = SpeechConfig.FromEndpoint(uri, _options.Value.Subscription);
config.SpeechRecognitionLanguage = language;
return config;

The code using config is the same, the first one returns error, the second one is working fine.

Any ideas how to fix that?

Answer 1

I've found an answer here (maybe not directly, but it gave me some ideas):

https://github.com/Azure-Samples/cognitive-services-speech-sdk/issues/2552

In order to connect to the private endpoint to use speech to text service, you have to specify the endpoint manually, the same way as using the Endpoint + key authentication, e.g.:

var uri = new Uri($"wss://{domain}/stt/speech/recognition/conversation/cognitiveservices/v1");

This is mentioned here:

https://learn.microsoft.com/en-us/azure/ai-services/speech-service/speech-services-private-link?tabs=portal

In order to authenticate using Microsoft Entra, just pass the received AAD token to the AuthorizationToken property, e.g.

config.AuthorizationToken = "my aad token";

Do not modify the authorization token like mentioned here:

https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-configure-azure-ad-auth?tabs=portal&pivots=programming-language-csharp
var authorizationToken = $"aad#{resourceId}#{aadToken}";

Finally the code that is working for me is as follows:

var tokenRequest = new TokenRequestContext(["https://cognitiveservices.azure.com/.default"]);
var azureCredential = new WorkloadIdentityCredential();
var aadToken = await azureCredential.GetTokenAsync(tokenRequest, default);
var endpoint = "my-private-endpoint uri";
var domain = new Uri(endpoint).DnsSafeHost;
var uri = new Uri($"wss://{domain}/stt/speech/recognition/conversation/cognitiveservices/v1");

var config = SpeechConfig.FromEndpoint(uri);
config.AuthorizationToken = aadToken;

I hope this is helpful for anyone who struggles with authenticating with private endpoint using Microsoft Entra.

I hope the documentation will be updated to explain such case.