DLP for South African ID Number

Lethukuthula Shabalala 1

Hi All,

I need assistance in creating a functional Sensitive Info Type (SIT) to prevent the unintended or unauthorized sharing of South African ID numbers. I’ve tried using the existing SIT, but it doesn't detect the ID numbers during testing. Additionally, I attempted to create a custom SIT using a regular expression, and although I've found some expressions that are typically used to validate South African ID numbers, they aren't working as expected. The regular expressions work fine on https://regex101.com, but when I apply them in the SIT configuration, they fail.

Any guidance would be greatly appreciated!

vaibhav kamra 0 Reputation points

2024-11-20T12:40:07.4066667+00:00
When using custom SIT, some regular expressions might:

Fail due to unsupported constructs in Microsoft's regex engine.

Require additional context to reduce false positives or improve accuracy.

Create a Custom SIT

Follow these steps to create a custom SIT to detect South African ID numbers:

Step 1: Build a Simplified Regex

Start with a regex tailored for Microsoft's SIT capabilities:

Basic Regex: \b\d{2}(0[1-9]|1[0-2])(0[1-9]|[12][0-9]|3[01])\d{4}[01]8\d\b

This regex checks:

13 digits total.

The date portion (YYMMDD) matches valid months and days.

Citizenship (C) is either 0 or 1.

Reserved field (A) is 8.

Add a Supporting Element

Microsoft SIT often requires confidence boosters (keywords, patterns) to enhance detection accuracy. Examples:

Keywords: "South African ID", "RSA ID", "ZA ID number".

Proximity: Ensure these keywords are near the potential ID numbers (e.g., within 50 characters).

Configure the Custom SIT

Navigate to Microsoft Purview Compliance Portal:

Go to Compliance > Data Classification > Sensitive Info Types.

Click Create a sensitive info type.

Define the SIT:

Name: South African ID

Description: Custom SIT to detect South African ID numbers.

Add the Regular Expression:

In the Pattern section, add the regex: \b\d{2}(0[1-9]|1[0-2])(0[1-9]|[12][0-9]|3[01])\d{4}[01]8\d\b

Set confidence level to medium or high based on testing.

Add Keywords as Supporting Elements:

Include keywords like "South African ID" or "ZA ID" as context.

Set Proximity:

Configure proximity for keywords (e.g., 50 characters).

Save and Publish.When using custom SIT, some regular expressions might:

Fail due to unsupported constructs in Microsoft's regex engine.

Require additional context to reduce false positives or improve accuracy.

3. Create a Custom SIT Follow these steps to create a custom SIT to detect South African ID numbers: Step 1: Build a Simplified Regex Start with a regex tailored for Microsoft's SIT capabilities:

Basic Regex: \b\d{2}(0[1-9]|1[0-2])(0[1-9]|[12][0-9]|3[01])\d{4}[01]8\d\b

This regex checks:

13 digits total.

The date portion (YYMMDD) matches valid months and days.

Citizenship (C) is either 0 or 1.

Reserved field (A) is 8.

Step 2: Add a Supporting Element Microsoft SIT often requires confidence boosters (keywords, patterns) to enhance detection accuracy. Examples:

Keywords: "South African ID", "RSA ID", "ZA ID number".

Proximity: Ensure these keywords are near the potential ID numbers (e.g., within 50 characters).

Step 3: Configure the Custom SIT

Navigate to Microsoft Purview Compliance Portal:

Go to Compliance > Data Classification > Sensitive Info Types.

Click Create a sensitive info type.

Define the SIT:

Name: South African ID

Description: Custom SIT to detect South African ID numbers.

Add the Regular Expression:

In the Pattern section, add the regex:
\b\d{2}(0[1-9]|1[0-2])(0[1-9]|[12][0-9]|3[01])\d{4}[01]8\d\b

Set confidence level to medium or high based on testing.

Add Keywords as Supporting Elements:

Include keywords like "South African ID" or "ZA ID" as context.

Set Proximity:

Configure proximity for keywords (e.g., 50 characters).

Save and Publish.

Let me know if any extra help is needed
phemanth 11,470 Reputation points Microsoft Vendor

2024-11-22T19:03:18.8+00:00

@Lethukuthula Shabalala We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Lethukuthula Shabalala 1 Reputation point

2024-11-23T14:31:09.04+00:00

Thank you for your response. But, the provided solution appears to be repeated. No offense intended, but it seems like a suggested solution from ChatGPT or a similar AI tool. I did attempt the mentioned regex before reaching out for further assistance.

After further investigation, I discovered that Microsoft Purview utilizes a subset of .NET regex, which can cause certain features to behave differently. For instance, I had to avoid using \b (word boundary) and instead opted for (?<!\d) as an opener and (?!\d) as a closer for the regex.

Here’s an example comparing the original regex we use on Mimecast and the modified version for Purview:

Original Regex (Mimecast):

\b(([0-57-9]\d(0[1-9]|1[012]))|(61-9)|(601[012]))(0[1-9]|[12][0-9]|3[01])[ -]?\d\d\d\d[ -]?\d\d\d\b

Modified Regex (Purview):

(?<!\d)(([0-57-9]\d(0[1-9]|1[0-2]))|(61-9)|(601[0-2]))(0[1-9]|[12][0-9]|3[01])[ -]?\d{4}[ -]?\d{3}(?!\d)

The modified regex successfully worked in my SIT environment and can distinguish between valid and invalid ID numbers.

Thank you once again, and I appreciate the community's continued support.

Share via

DLP for South African ID Number

Your answer