How to detect and redact Personally Identifying Information (PII)
The PII feature can evaluate unstructured text, extract and redact sensitive information (PII) and health information (PHI) in text across several pre-defined categories.
Development options
To use PII detection, you submit text for analysis and handle the API output in your application. Analysis is performed as-is, with no customization to the model used on your data. There are two ways to use PII detection:
| Development option | Description |
|---|---|
| Language studio | Language Studio is a web-based platform that lets you try entity linking with text examples without an Azure account, and your own data when you sign up. For more information, see the Language Studio website or language studio quickstart. |
| REST API or Client library (Azure SDK) | Integrate PII detection into your applications using the REST API, or the client library available in various languages. For more information, see the PII detection quickstart. |
Determine how to process the data (optional)
Specify the PII detection model
By default, this feature uses the latest available AI model on your text. You can also configure your API requests to use a specific model version.
Input languages
When you submit documents to be processed, you can specify which of the supported languages they're written in. if you don't specify a language, extraction defaults to English. The API may return offsets in the response to support different multilingual and emoji encodings.
Redaction Policy (version 2024-11-5-preview only)
in version 2024-11-5-preview, you're able to define the redactionPolicy parameter to reflect the redaction policy to be used when redacting the document in the response. The policy field supports 3 policy types:
DoNotRedactMaskWithCharacter(default)MaskWithEntityType
The DoNotRedact policy allows the user to return the response without the redactedText field.
The MaskWithRedactionCharacter policy allows the redactedText to be masked with a character, preserving the length and offset of the original text. This is the existing behavior.
There is also an optional field called redactionCharacter where you can input the character to be used in redaction if you're using the MaskWithCharacter policy
The MaskWithEntityType policy allows you to mask the detected PII entity text with the detected entity type.
Submitting data
Analysis is performed upon receipt of the request. Using the PII detection feature synchronously is stateless. No data is stored in your account, and results are returned immediately in the response.
When using this feature asynchronously, the API results are available for 24 hours from the time the request was ingested, and is indicated in the response. After this time period, the results are purged and are no longer available for retrieval.
Select which entities to be returned
The API attempts to detect the defined entity categories for a given document language. If you want to specify which entities are detected and returned, use the optional piiCategories parameter with the appropriate entity categories. This parameter can also let you detect entities that aren't enabled by default for your document language. The following example would detect only Person. You can specify one or more entity types to be returned.
Tip
If you don't include default when specifying entity categories, The API only returns the entity categories you specify.
Input:
Note
In this example, it returns only the person entity type:
https://<your-language-resource-endpoint>/language/:analyze-text?api-version=2022-05-01
{
"kind": "PiiEntityRecognition",
"parameters":
{
"modelVersion": "latest",
"piiCategories" :
[
"Person"
]
},
"analysisInput":
{
"documents":
[
{
"id":"1",
"language": "en",
"text": "We went to Contoso foodplace located at downtown Seattle last week for a dinner party, and we adore the spot! They provide marvelous food and they have a great menu. The chief cook happens to be the owner (I think his name is John Doe) and he is super nice, coming out of the kitchen and greeted us all. We enjoyed very much dining in the place! The pasta I ordered was tender and juicy, and the place was impeccably clean. You can even pre-order from their online menu at www.contosofoodplace.com, call 112-555-0176 or send email to order@contosofoodplace.com! The only complaint I have is the food didn't come fast enough. Overall I highly recommend it!"
}
]
},
"kind": "PiiEntityRecognition",
"parameters": {
"redactionPolicy": {
"policyKind": "MaskWithCharacter"
//MaskWithCharacter|MaskWithEntityType|DoNotRedact
"redactionCharacter": "*"
}
Output:
{
"kind": "PiiEntityRecognitionResults",
"results": {
"documents": [
{
"redactedText": "We went to Contoso foodplace located at downtown Seattle last week for a dinner party, and we adore the spot! They provide marvelous food and they have a great menu. The chief cook happens to be the owner (I think his name is ********) and he is super nice, coming out of the kitchen and greeted us all. We enjoyed very much dining in the place! The pasta I ordered was tender and juicy, and the place was impeccably clean. You can even pre-order from their online menu at www.contosofoodplace.com, call 112-555-0176 or send email to order@contosofoodplace.com! The only complaint I have is the food didn't come fast enough. Overall I highly recommend it!",
"id": "1",
"entities": [
{
"text": "John Doe",
"category": "Person",
"offset": 226,
"length": 8,
"confidenceScore": 0.98
}
],
"warnings": []
}
],
"errors": [],
"modelVersion": "2021-01-15"
}
}
Getting PII results
When you get results from PII detection, you can stream the results to an application or save the output to a file on the local system. The API response includes recognized entities, including their categories and subcategories, and confidence scores. The text string with the PII entities redacted is also returned.
Service and data limits
For information on the size and number of requests you can send per minute and second, see the service limits article.