How to get pipeline (View profiling) detail through Azure ML SDK in Python?

Question

Hi team,

I'm struggling to figure out how we can get the pipeline profiling detail.

Can anyone please guide

from azure.ai.ml import MLClient
from azure.identity import DefaultAzureCredential

# Initialize the MLClient
ml_client = MLClient(
     DefaultAzureCredential(),
     subscription_id="<Sub_id>",
     resource_group_name="<Resource_group>",
    workspace_name="<Workspace_nmae>"
 )

# Get job details
job_details = ml_client.jobs.get(name="purple_forest_lfzpft1y51")

Answer 1

Hi Puneet

Welcome to Microsoft Q&A Forum, thank you for posting your query here!

Profiling option of pipelines outputs overall pipeline performance details with respect to duration of each step in Gantt chart.

I could not find or come with exact SDK documentation right away, but we will have to get all child steps and extract their details like name, duration etc.

Here is sample SDK snippet drafted on the basis of pipeline input and output documentation

from azure.ai.ml import MLClient
from azure.identity import DefaultAzureCredential
from azure.ai.ml.entities import Job
# Authenticate and create ML client
credential = DefaultAzureCredential()
ml_client = MLClient(credential, subscription_id, resource_group, workspace_name)
# Get the details of the pipeline job
pipeline_job = ml_client.jobs.get(pipeline_job.name)
# Fetch the job's child runs (each step in the pipeline)
child_jobs = ml_client.jobs.list(parents=pipeline_job.name)
# Print the name and duration of each step
for child_job in child_jobs:
    print(f"Step Name: {child_job.name}")
    print(f"Start Time: {child_job.start_time}")
    print(f"End Time: {child_job.end_time}")
    duration = child_job.end_time - child_job.start_time
    print(f"Duration: {duration}\n")

Hope it helps.

If this answers your query, do click **Accept Answer **and Yes for was this answer helpful.

Thank you.