I'm using real-time Azure AI Speech recognition with automatic language recognition. I would like to filter out low-confidence recognition results, which mostly caused by background noise or unclear microphone input. I'm using Node.js SDK. It seems there are two fields - confidence and language recognition confidence. Confidence - It seems not exposed on Node.js API. Is there any workaround? Language confidence - While it's exposed, its value is always 0. Can you help me with this?

Hi Gunwoo Kim , Thank you for reaching out to Microsoft Q&A forum! Yes, you are correct that the confidence score of speech recognition results is not directly exposed in the Azure AI Speech SDK for Node.js. However, there are some workarounds and best practices you can use to filter out low-confidence results. Confidence Score: Azure AI Speech SDK (JavaScript/Node.js) does not currently expose the confidence score of the recognized speech result. This is a limitation compared to the Python or C# SDK, which does provide confidence scores. Language Recognition Confidence: This field is exposed but is always returning 0, which seems like an issue with how it's being retrieved or a limitation in the API response. Workarounds: Using Detailed Recognition Mode (JSON Response): Although the confidence score is not directly available in the Node.js SDK, you can enable detailed output and extract it manually. Steps to Enable Confidence Scores in JSON Output: Use speech config with setProperty to enable detailed JSON output. Parse the JSON response to extract the confidence score. The jsonResult contains detailed recognition results, including confidence scores. Extract NBest[0].Confidence (highest-ranked hypothesis). Apply filtering: Accept results only if confidence > 0.7. Please refer to the below document - Node.js speech SDK attaching for your reference. https://learn.microsoft.com/en-us/azure/ai-services/speech-service/speech-sdk Hope this helps. Do let us know if you any further queries. ------------- If this answers your query, do click Accept Answer and Yes for was this answer helpful. Thank you.

How to access confidence score of speech recognition results in JavaScript

Accepted answer

Prashanth Veeragoni 640 Reputation points Microsoft Vendor

2025-02-20T05:28:08.9466667+00:00

Hi Gunwoo Kim,

Thank you for reaching out to Microsoft Q&A forum!

Yes, you are correct that the confidence score of speech recognition results is not directly exposed in the Azure AI Speech SDK for Node.js. However, there are some workarounds and best practices you can use to filter out low-confidence results.

Confidence Score: Azure AI Speech SDK (JavaScript/Node.js) does not currently expose the confidence score of the recognized speech result.

This is a limitation compared to the Python or C# SDK, which does provide confidence scores.

Language Recognition Confidence: This field is exposed but is always returning 0, which seems like an issue with how it's being retrieved or a limitation in the API response.

Workarounds:

Using Detailed Recognition Mode (JSON Response): Although the confidence score is not directly available in the Node.js SDK, you can enable detailed output and extract it manually.

Steps to Enable Confidence Scores in JSON Output:

Use speech config with setProperty to enable detailed JSON output.

Parse the JSON response to extract the confidence score.

The jsonResult contains detailed recognition results, including confidence scores.

Extract NBest[0].Confidence (highest-ranked hypothesis).

Apply filtering: Accept results only if confidence > 0.7.

Please refer to the below document - Node.js speech SDK attaching for your reference.

https://learn.microsoft.com/en-us/azure/ai-services/speech-service/speech-sdk

Hope this helps. Do let us know if you any further queries.

-------------

If this answers your query, do click Accept Answer and Yes for was this answer helpful.

Thank you.
Please sign in to rate this answer.
Prashanth Veeragoni 640 Reputation points Microsoft Vendor

2025-02-21T01:28:01.0533333+00:00

Hi Gunwoo Kim,

Following up to see if the above answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let me know.

Thank You.

Gunwoo Kim 20 Reputation points

2025-02-21T02:45:14.9333333+00:00

Thanks for getting back. I tried your solution, but it seems that the value of NBest[0].Confidence is always zero. Below are the examples.

{"Id":"8e142545118d4cc282a818992eaaa6be","RecognitionStatus":"Success","Offset":275700000,"Duration":20800000,"PrimaryLanguage":{"Language":"ja-JP","Confidence":"Unknown"},"Channel":0,"DisplayText":"ここから、日本語で話します。","NBest":[{"Confidence":0,"Lexical":"ここから日本語で話します","ITN":"ここから日本語で話します","MaskedITN":"ここから、日本語で話します","Display":"ここから、日本語で話します。"}]}"

{"Id":"da4facbf59644e4fa7ff5c8f18546406","Text":"confidence","Offset":458300000,"Duration":23500000,"PrimaryLanguage":{"Language":"en-US"},"Channel":0}"{"Id":"218bca4f447c403e903e5682d4989ec5","RecognitionStatus":"Success","Offset":458300000,"Duration":26800000,"PrimaryLanguage":{"Language":"en-US","Confidence":"Unknown"},"Channel":0,"DisplayText":"Confidence value is always 0.","NBest":[{"Confidence":0,"Lexical":"confidence value is always zero","ITN":"confidence value is always 0","MaskedITN":"confidence value is always 0","Display":"Confidence value is always 0."}]}"

Gunwoo Kim 20 Reputation points

2025-02-21T02:47:25.3233333+00:00

Thanks for getting back. I tried your solution but it seems that the value of NBest[0].Confidence is always zero. Below are the examples.

{\"Id\":\"8e142545118d4cc282a818992eaaa6be\",\"RecognitionStatus\":\"Success\",\"Offset\":275700000,\"Duration\":20800000,\"PrimaryLanguage\":{\"Language\":\"ja-JP\",\"Confidence\":\"Unknown\"},\"Channel\":0,\"DisplayText\":\"ここから、日本語で話します。\",\"NBest\":[{\"Confidence\":0,\"Lexical\":\"ここから日本語で話します\",\"ITN\":\"ここから日本語で話します\",\"MaskedITN\":\"ここから、日本語で話します\",\"Display\":\"ここから、日本語で話します。\"}]}"

{\"Id\":\"218bca4f447c403e903e5682d4989ec5\",\"RecognitionStatus\":\"Success\",\"Offset\":458300000,\"Duration\":26800000,\"PrimaryLanguage\":{\"Language\":\"en-US\",\"Confidence\":\"Unknown\"},\"Channel\":0,\"DisplayText\":\"Confidence value is always 0.\",\"NBest\":[{\"Confidence\":0,\"Lexical\":\"confidence value is always zero\",\"ITN\":\"confidence value is always 0\",\"MaskedITN\":\"confidence value is always 0\",\"Display\":\"Confidence value is always 0.\"}]}"

Prashanth Veeragoni 640 Reputation points Microsoft Vendor

2025-02-21T05:13:07.9366667+00:00

Hi Gunwoo Kim,

Thank you for your patience! I investigated the issue, and here’s the reason why confidence values are always 0 in the Node.js SDK:

The Node.js SDK does not expose confidence scores, unlike Python or C#.

The PrimaryLanguage.Confidence field often returns "Unknown", meaning the language detection process does not provide meaningful confidence values.

Even with detailed JSON output enabled, the NBest[0].Confidence field is always 0 in Node.js, making it unreliable for filtering results.

Workarounds:

Use Python or C# SDK (Best Option):

These SDKs properly support confidence scores.

If switching SDKs is an option, this will solve the issue completely.

Use a Custom Speech Model:

A custom model improves recognition quality and reduces background noise errors.

You can train a custom model via Azure Speech Studio:
https://learn.microsoft.com/en-us/azure/ai-services/speech-service/custom-speech-overview

Apply Noise Suppression Before Sending Audio:

Pre-process audio using WebRTC Noise Suppression in Node.js or PyDub in Python to reduce unwanted noise before sending audio for recognition.

Unfortunately, confidence filtering is not directly possible in Node.js, but these solutions should help you get more accurate results. Let me know if you need further assistance!

Could you please take a moment to retake the survey on the above response? Your feedback is greatly appreciated.

Thank You.

Prashanth Veeragoni 640 Reputation points Microsoft Vendor

2025-02-24T00:55:52.8+00:00

Hi Gunwoo Kim,

Following up to see if the above answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let me know.

Thank You.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

How to access confidence score of speech recognition results in JavaScript

0 additional answers

Your answer