Problem scanning Data Sources in Purview

Question

Hi!

We are currently trying to set up purview for our organization. And whenever we are trying to scan a data source we get an ambigious "internal server error". Note that when we try to "test connection", there is no problem. This problem arises for every type of data source, in our case we have tried to scan Analytic lake (Storage V2), Power BI, Azure dedicated pools (our DWH), and an SQL database.

If we use Analytic lake as an example; the connection is successfully established as mentioned, we can see all containers and the associated blobs before we start the scan.

We use:

• Azure AutoresolveintegrationRuntime
• Microsoft Purview-MSI (system) for authentication

Purview also has contributor role on resource group level and Storage Blob Data Reader on the storage account for analytic lake.

We have also double checked our firewall settings for the resource group we are trying to access and there seem to be no restriction there from what we can see.

Any idea what the issue could be?

Answer 1

@Love Book

Please Check the below steps below steps and confirm us

1. Check Integration Runtime (IR) Version: Ensure that you are using the latest version of the Azure Integration Runtime. If you're using a Self-hosted Integration Runtime (SHIR), updating it to the latest version might resolve the issue
2. Review Scan Logs: Go to the 'Management Center' > 'Scans' > select the scan which has failed. This should provide more detailed information about the error. Look for any specific error messages or logs that might give more insight into why the scan is failing
3. Verify Permissions: Double-check that the Microsoft Purview Managed Identity (MSI) has the necessary permissions. It should have the 'Contributor' role at the resource group level and 'Storage Blob Data Reader' on the storage account for the Analytic Lake
4. Network and Firewall Settings: Even though you've checked the firewall settings, ensure there are no network restrictions or policies that might be blocking the scan. Sometimes, network issues can cause intermittent errors
5. Retry the Scan: Sometimes, the error might be temporary. Try running the scan again after some time to see if the issue persists

Please refer : https://learn.microsoft.com/en-us/purview/troubleshoot-connections

I hope this information helps. Please do let us know if you have any further queries.