How to fix error trying to set up data profiling in Purview :"Unable to submit job to PFS with exception: The given key 'northcentralus' was not present in the dictionary.

Question

How to fix error trying to set up data profiling in Purview :"Unable to submit job to PFS with exception: The given key 'northcentralus' was not present in the dictionary.

Harrison, Peter K 0

Our Purview is in South Central US, our ADLS-gen2 account is in North Central US. We have set up a connection from Purview to ADLS Gen2 - when we run profile on a parquet file, the job is accepted, but eventually fails with the message: "Unable to submit job to PFS with exception: The given key 'northcentralus' was not present in the dictionary."

If we retry setting up the connection with Enable managed V-Net and look for North Central US as a Region it is not in the list. If we choose South Central US we get a status code 500.

Chandra Boorla 9,985 Reputation points Microsoft External Staff

2025-03-05T19:52:20.4433333+00:00
Hi @Harrison, Peter K

Thank you for posting your query!

The error message "Unable to submit job to PFS with exception: The given key 'northcentralus' was not present in the dictionary" suggests that there may be an issue with the region configuration between your Microsoft Purview account and the Azure Data Lake Storage (ADLS) Gen2 account. Since your Purview is in South Central US and the ADLS Gen2 account is in North Central US, this mismatch could be causing the error.

To resolve this issue, consider the following steps:

Region Compatibility - Ensure that both your Purview account and ADLS Gen2 account are in compatible regions. If North Central US is not available as a region option when setting up the connection, it may indicate that cross-region profiling is not supported.

Managed V-Net Configuration - When enabling managed V-Net, ensure that the selected region is compatible with both Purview and ADLS. If you receive a status code 500 when selecting South Central US, it may indicate a temporary issue or misconfiguration.

Check Documentation - Review the Microsoft documentation for any specific requirements or limitations regarding region configurations for data profiling in Purview.

You can check the Azure Purview documentation for updates on the availability of Purview in different regions.

For additional information, please refer:

Reliability in Microsoft Purview

Supported regions for Microsoft Purview Data Governance

Configure and run data profiling for a data asset

I hope this information helps. Please do let us know if you have any further queries.

Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues.

Thank you.
Chandra Boorla 9,985 Reputation points Microsoft External Staff

2025-03-06T18:36:43.58+00:00

@Harrison, Peter K

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution, please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Harrison, Peter K 0 Reputation points

2025-03-06T19:28:05.25+00:00

This does not seem to specifically address our problem.
Rakesh Govindula 95 Reputation points Microsoft External Staff

2025-03-07T09:33:18.68+00:00

Hello @Harrison, Peter K,

As per this MS Documentation,

Currently Azure Purview is not supported in South Central US Region.

This might be the reason for the above behavior.

You can cross-check it with a sample parquet data in the same region, if it works then you can confirm it as region incompatibility in North Central US.

To achieve the cross-region Profile scan in Azure Purview, you can use Self hosted Integration run time on your region. Set up a SHIR on a virtual machine located in the desired region, then register it with your Azure Purview account. It will act as bridge between these regions.

Reference: Create and manage a self-hosted integration runtime

I hope this information helps. Please do let us know if you have any further queries.

Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues.

2 answers

Your answer

Chandra Boorla 9,985 Reputation points Microsoft External Staff

2025-03-05T19:52:20.4433333+00:00

Hi @Harrison, Peter K

Thank you for posting your query!

The error message "Unable to submit job to PFS with exception: The given key 'northcentralus' was not present in the dictionary" suggests that there may be an issue with the region configuration between your Microsoft Purview account and the Azure Data Lake Storage (ADLS) Gen2 account. Since your Purview is in South Central US and the ADLS Gen2 account is in North Central US, this mismatch could be causing the error.

To resolve this issue, consider the following steps:

Region Compatibility - Ensure that both your Purview account and ADLS Gen2 account are in compatible regions. If North Central US is not available as a region option when setting up the connection, it may indicate that cross-region profiling is not supported.

Managed V-Net Configuration - When enabling managed V-Net, ensure that the selected region is compatible with both Purview and ADLS. If you receive a status code 500 when selecting South Central US, it may indicate a temporary issue or misconfiguration.

Check Documentation - Review the Microsoft documentation for any specific requirements or limitations regarding region configurations for data profiling in Purview.

You can check the Azure Purview documentation for updates on the availability of Purview in different regions.

For additional information, please refer:

Reliability in Microsoft Purview

Supported regions for Microsoft Purview Data Governance

Configure and run data profiling for a data asset

I hope this information helps. Please do let us know if you have any further queries.

Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues.

Thank you.
Chandra Boorla 9,985 Reputation points Microsoft External Staff

2025-03-06T18:36:43.58+00:00

@Harrison, Peter K

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution, please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Harrison, Peter K 0 Reputation points

2025-03-06T19:28:05.25+00:00

This does not seem to specifically address our problem.
Rakesh Govindula 95 Reputation points Microsoft External Staff

2025-03-07T09:33:18.68+00:00

Hello @Harrison, Peter K,

As per this MS Documentation,

Currently Azure Purview is not supported in South Central US Region.

This might be the reason for the above behavior.

You can cross-check it with a sample parquet data in the same region, if it works then you can confirm it as region incompatibility in North Central US.

To achieve the cross-region Profile scan in Azure Purview, you can use Self hosted Integration run time on your region. Set up a SHIR on a virtual machine located in the desired region, then register it with your Azure Purview account. It will act as bridge between these regions.

Reference: Create and manage a self-hosted integration runtime

I hope this information helps. Please do let us know if you have any further queries.

Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues.

Answer 1

Harrison, Peter K 0

Purview IS supported in South Central US, not in North Central US. We have a windows self-hosted runtime working for catalog scans, but the documentation for profile scans seems to indicate that is not supported for Data quality scanning.

AnnuKumari-MSFT 34,351 Reputation points Microsoft Employee

2025-03-10T09:41:08.5333333+00:00
Hi Harrison, Peter K ,

You are right . Microsoft Purview is supported in South central US , but not in North central US.

Did you try moving ADLS gen2 to South centra US? The issue looks related to region pairing and VNet integration issue between Microsoft Purview and Azure Data Lake Storage Gen2.

When you pick South Central US instead (where Purview lives), it can’t access the data source in North Central US, which might have resulted in the above error. This means Purview's job submission logic isn't finding or handling northcentralus correctly because PFS doesn't have an endpoint there.

You can go ahead with trying one of the below possibilities:

Move ADLS to South Central US

Try disabling Managed VNet on Purview, and allow public network access temporarily (if security allows), then retry profiling.

Hope it helps. Kindly let us know how it goes. Thankyou
AnnuKumari-MSFT 34,351 Reputation points Microsoft Employee

2025-03-11T09:31:59.34+00:00

Hi Harrison, Peter K ,

Did you try the above suggested approach? Please let me know how it goes. Thankyou

Answer 2

Harrison, Peter K 0

I am not authorized to do either of those actions - they violate our security policies and my role/capabilities. Better that you make Purview available in North Central US region immediately.

Share via

How to fix error trying to set up data profiling in Purview :"Unable to submit job to PFS with exception: The given key 'northcentralus' was not present in the dictionary.

2 answers

Your answer