How to restrict sensitive information in AI Search

Question

Hi Team,

How can i restrict sensitive information while i querying from AI Search.

Scenario : If Credit card information is there in a document , it should n't retrieve while user asked in his prompt.

How can i acheive this while using AI Search in c#

Thanks

Answer 1

Hi @Balaji Mogadali,
Welcome to the Microsoft Q&A Platform!
To prevent sensitive information from being retrieved during searches, implement the following two-step approach:

PII Detection:

1. Make use of Azure AI's PII (Personally Identifiable Information) detection feature. Scan your documents for names, addresses, social security numbers, etc. You can make it happen using Azure AI Text Analytics client library within your C# application.
2. Once the PII entities have been identified, you may entirely remove them or mask them and then index it into your search index. It prevents the information from reaching users in the returned search results.
   The below is a simplified example of how you would implement PII detection and redaction in your C# application:

using Azure;
using Azure.AI.TextAnalytics;
public class PiiRedactionService
{
    private readonly TextAnalyticsClient _textAnalyticsClient;
    public PiiRedactionService(string endpoint, string apiKey)
    {
        var credentials = new AzureKeyCredential(apiKey);
        _textAnalyticsClient = new TextAnalyticsClient(new Uri(endpoint), credentials);
    }
    public string RedactSensitiveInformation(string document)
    {
        var response = _textAnalyticsClient.RecognizePiiEntities(document);
        var redactedText = response.Value.RedactedText;
        return redactedText;
    }
}

Query-Time Filtering:

1. During designing the structure of your search index in C#, you must make sure to set the IsRetrievable property of sensitive fields to false. This is to say that these fields are not retrievable in search results, even though a user searched for them.
2. When users perform searches, use filters to specifically exclude any documents that contain sensitive data. For example, you can use the $filter parameter to create rules that remove documents based on certain conditions, ensuring that sensitive information is never returned in the search results.

Overview:
https://learn.microsoft.com/en-us/azure/search/cognitive-search-skill-pii-detection

Hope the above provided information help in better understanding and help you resolve the issue, if you have any further concerns or queries, please feel free to reach out to us.
If the answer is helpful, please click Accept Answer and kindly upvote it so that other people who faces similar issue may get benefitted from it.