Unsure of how to to use pii detection cognitive skill azure ai resource skills

Question

Hello,

I'm quite new to azure so apologies in advance if none of this makes sense. I am having some difficulties with the pii cognitive skill on azure ai (https://learn.microsoft.com/en-us/azure/search/cognitive-search-skill-pii-detection). I was able to follow the quick start tutorial but on the azure ai language skill for pii (https://learn.microsoft.com/en-us/azure/ai-services/language-service/personally-identifiable-information/quickstart?tabs=windows&pivots=programming-language-python) but for some reason even though I followed the exact same steps and tried with the same example as provided in the first link, I just couldn't get it to work - I never was able to detect the pii entities. So my question is how do I address this problem. I assume its not an issue with indexing as I used the indexer and was able to get the content I put in my data source into the index. Is it an issue with how I set up the template for the skill which I find it unlikely as I followed the sample provided. Additionally, is it possible to use the second tutorial and kind of deploy that as a skill onto azure portal? Thanks in advance.

Answer 1

Hi Muhammad Abdulbaqi,

Thanks for reaching out to Microsoft Q&A.

For using Azure’s PII detection cognitive skill within Azure Cognitive Search, there are a few troubleshooting and setup steps you can check, and I’ll address deploying it as well:

1. Ensure Proper Skill Configuration in the Skillset Template: Confirm that your skillset template is set up to reference the piiEntities skill correctly. The configuration should match the one in the tutorial exactly, particularly with fields like defaultLanguageCode and categories. Verify that any settings or parameters in the JSON configuration for piiEntities match the tutorial, as even small discrepancies can sometimes lead to unexpected issues.

Skillset Pipeline Validation: Sometimes, misconfigurations or connection issues can prevent the skillset from detecting PII. To troubleshoot, navigate to the Cognitive Search skillset in the portal and validate the pipeline’s execution by running a sample document through the skillset and reviewing output logs or errors in the Cognitive Search diagnostic logs.

Verify Permissions and Resource Configuration: Ensure that the Azure Cognitive Search resource and any linked Azure AI resource have proper permissions and are within the same region. Cross-region calls may lead to issues if networking settings aren’t aligned. Also, check that the language resource has been deployed in the correct region with the PII capability enabled.

Using the Azure Language Service as a Standalone Skill: If you prefer working directly with Azure Language Service for PII detection (like in the second tutorial), you can indeed deploy this as a custom skill in Cognitive Search. You would set up a custom Web API skill that calls the PII endpoint in Azure Language Service. This involves creating an Azure Function (or other endpoint) that calls the PII detection service, then configuring this endpoint as a custom skill in your Cognitive Search skillset.

To summarize, check configuration and diagnostic logs, ensure matching setup between your Cognitive Search skillset and Azure Language Service, and explore custom Web API skills if you want direct integration with the Language Service PII endpoint.

Please feel free to click the 'Upvote' (Thumbs-up) button and 'Accept as Answer'. This helps the community by allowing others with similar queries to easily find the solution.