After using api to purge data from workspace the status stays on pending and logs are not removed.

Question

After using api to purge data from workspace the status stays on pending and logs are not removed.

While using the status endpoint the status keeps returning as 'pending'.

The purge request is visible in activity logs of the workspace but just shows as Accepted -> Started.

Answer 1

Hi Danny Baatje,

Thanks for reaching out to Microsoft Q&A.

The issue you're experiencing, where the data purge request in azure synapse remains in a "pending" status and the logs are not removed is likely related to the purge job not completing as expected. There are a few potential reasons for this and steps you can take to investigate and resolve the issue. Steps to troubleshoot and resolve:

1. Verify Permissions:
   • Ensure that the API call to purge data is being made by a user or service principal that has the necessary permissions to perform the purge operation. This includes proper access roles for managing data within Synapse.
2. Check the Synapse Pipeline and Integration Runtime:
   • If the purge operation depends on any Synapse pipelines or IR, ensure that the IR is running and properly configured.
   • Confirm that no other jobs are currently running that might be blocking the purge request.
3. Examine Quotas and Resource Availability:
   • Verify if there are any quotas or resource limits (like concurrency) that might be preventing the purge request from proceeding.
   • You can check resource limits in the Azure portal under Synapse workspace -> Quotas to ensure the purge is not being throttled.
4. Check for Pending Dependencies:
   • There might be dependencies on the data being purged that are preventing it from being processed. Review the workspace for any locks or ongoing processes that could be affecting the purge.
5. Activity Logs Analysis:
   • The activity log showing Accepted -> Started without further updates could mean that the purge job is stuck. You may want to check Azure Monitor or Log Analytics for deeper logs related to Synapse activities to identify any underlying failures or delays.
6. Check API Rate Limits:
   • If multiple purge requests are submitted simultaneously, they might hit API rate limits. Check if throttling is occurring by inspecting the API responses for rate limit errors.

By following these steps, you should be able to determine the root cause of why the purge operation remains in a pending state and how to resolve it.

Please feel free to click the 'Upvote' (Thumbs-up) button and 'Accept as Answer'. This helps the community by allowing others with similar queries to easily find the solution.