Find total activities run and vCore hours consumed in a month from all of my monthly pipeline

Question

How can I see the total vCore hours consumed in a month from all of my pipeline runs inside a Synapse Workspace. I know how to get that value for each pipeline run but I am trying to find out whether there is a report/view/screen where it can be seen on aggregate, please?

Most of our pipelines use Mapping Data Flow and are executed through Synapse Pipelines. We also have some Synapse pipelines (Copy Data).

Purpose: To obtain the total cost of our pipeline runs and to be able to estimate monthly costs.

I have also seen in the past people have used PowerShell for getting vCore hours for ADF. Does anyone have a similar solution for Synapse?

Thanks in advance

Answer 1

Thanks for posting your question in the Microsoft Q&A forum.

To effectively manage and monitor your Azure Synapse Analytics usage and costs, start by navigating to the Azure portal and accessing the Cost Management section for your Synapse Workspace. Here, you can view a detailed breakdown of usage for Azure Synapse Analytics - Pipeline and Azure Synapse Analytics - Data Flow, which provides insights into total vCore hours consumed and associated costs. Next, set up Azure Monitor for your Synapse Workspace to create custom queries that aggregate vCore usage across all pipeline runs. Utilize Log Analytics to visualize and analyze this data for deeper insights. Within Synapse Studio, navigate to the "Monitor" hub to explore the "Pipeline runs" and "Data flow runs" sections. While these sections don’t directly display total vCore hours, they provide detailed execution information for each run. Additionally, you can adapt existing Azure Data Factory PowerShell scripts for Synapse Analytics by using the Azure Synapse Analytics PowerShell module. This allows you to query pipeline run history and calculate vCore hours programmatically.

---

Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful