Groundedness detection does not detect anything

Question

When I call the detect groundedness API in the Content Safety service, it always gives the same following response no matter the text or groundingSources:

{
    "ungroundedDetected": false,
    "ungroundedPercentage": 0,
    "ungroundedDetails": []
}

Here is my the call I make (example taken from the documentation):

curl --location 'MY_DOMAIN.cognitiveservices.azure.com/contentsafety/text:detectGroundedness?api-version=2024-09-15-preview' \
--header 'Host: MY_DOMAIN.cognitiveservices.azure.com' \
--header 'Content-Type: application/json' \
--header 'Ocp-Apim-Subscription-Key: MY_KEY \
--data '{
  "domain": "Generic",
  "task": "Summarization",
  "text": "The patient name is Kevin.",
  "groundingSources": [
    "The patient name is Jane."
  ]
}'

Answer 1

Hello Erol KOSEOGLU,

Welcome to the Microsoft Q&A and thank you for posting your questions here.

I understand that you're having issue with the Groundedness detection API not identifying any ungrounded text.

First thing to do is to double-check the Azure Content Safety documentation on groundedness - https://learn.microsoft.com/en-us/azure/ai-services/content-safety/concepts/groundedness for any recently added parameters, limitations, or updates specific to the preview version you are using.

Secondly, check that the text parameter is well-formed and coherent. Short or ambiguous texts may not trigger ungrounded detection effectively and verify that groundingSources aligns with the API's expectations. For instance, grounding sources should ideally be exhaustive and provide a complete context against which the text can be evaluated.

Thirdly, use more explicit differences between text and groundingSources to test if the API can detect ungrounded content. For example:

     {
       "text": "The patient's diagnosis is pneumonia.",
       "groundingSources": ["The patient's diagnosis is influenza."]
     }

Also, test with extreme examples where the text and groundingSources are clearly contradictory. For instance:

     {
       "text": "The sky is green.",
       "groundingSources": ["The sky is blue."]
     }

If no ungrounded detection occurs, this might indicate an issue with the API's configuration. You can use the below updated example for testing:

curl --location 'MY_DOMAIN.cognitiveservices.azure.com/contentsafety/text:detectGroundedness?api-version=2024-09-15-preview' \
--header 'Content-Type: application/json' \
--header 'Ocp-Apim-Subscription-Key: MY_KEY' \
--data '{
  "domain": "Generic",
  "task": "Summarization",
  "text": "The patient has been diagnosed with pneumonia.",
  "groundingSources": [
    "The patient has been diagnosed with influenza."
  ]
}'

The API may have thresholds for determining when something is "ungrounded." For example, subtle differences (like names) might not exceed the threshold for detection. Check if such thresholds are configurable or inherent to the service.

If the API consistently fails to detect ungrounded text, this could indicate a bug or misconfiguration in the service. Reach out to Azure support via your Azure Portal with specific details.

I hope this is helpful! Do not hesitate to let me know if you have any other questions.

---

Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful.