PublishedPipeline Class
Represents a Pipeline to be submitted without the Python code which constructed it.
In addition, a PublishedPipeline can be used to resubmit a Pipeline with different PipelineParameter values and inputs.
Initialize PublishedPipeline.
:param endpoint The REST endpoint URL to submit pipeline runs for this pipeline. :type endpoint: str :param total_run_steps: The number of steps in this pipeline :type total_run_steps: int :param workspace: The workspace of the published pipeline. :type workspace: azureml.core.Workspace :param continue_on_step_failure: Whether to continue execution of other steps in the PipelineRun
if a step fails, default is false.
- Inheritance
-
azureml.core._portal.HasPipelinePortalPublishedPipeline
Constructor
PublishedPipeline(name, graph_id, description, version, published_pipeline_id, status, endpoint, total_run_steps, workspace, continue_on_step_failure=None, _pipeline_provider=None, **kwargs)
Parameters
Name | Description |
---|---|
name
Required
|
The name of the published pipeline. |
graph_id
Required
|
The ID of the graph for this published pipeline. |
description
Required
|
The description of the published pipeline. |
version
Required
|
The published pipeline version. |
published_pipeline_id
Required
|
The ID of the published pipeline. |
status
Required
|
The status of the published pipeline ('Active' or 'Disabled'). |
endpoint
Required
|
The REST endpoint URL to submit runs for this pipeline. |
total_run_steps
Required
|
The number of steps in this pipeline. |
workspace
Required
|
The workspace of the published pipeline. |
continue_on_step_failure
Required
|
Whether to continue execution of other steps in the PipelineRun if a step fails. The default is false. |
_pipeline_provider
Required
|
<xref:azureml.pipeline.core._workflow_provider._PublishedPipelineProvider>
The published pipeline provider. |
kwargs
Required
|
Custom keyword arguments, reserved for future development |
name
Required
|
The name of the published pipeline. |
graph_id
Required
|
The ID of the graph for this published pipeline. |
description
Required
|
The description of the published pipeline. |
version
Required
|
The published pipeline version. |
published_pipeline_id
Required
|
The ID of the published pipeline. |
status
Required
|
Status of the published pipeline ('Active' or 'Disabled'). |
_pipeline_provider
Required
|
<xref:azureml.pipeline.core._workflow_provider._PublishedPipelineProvider>
The published pipeline provider. |
kwargs
Required
|
Custom keyword arguments, reserved for future development |
Remarks
A PublishedPipeline can be created from either a Pipeline or a PipelineRun.
An example to publish from a Pipeline is as follows:
from azureml.pipeline.core import Pipeline
pipeline = Pipeline(workspace=ws, steps=steps)
published_pipeline = pipeline.publish(name="My_New_Pipeline",
description="My New Pipeline Description",
version="1.0",
continue_on_step_failure=True)
To publish from a PipelineRun use:
from azureml.pipeline.core import PipelineRun
pipeline_run = PipelineRun(experiment=Experiment(ws, "Pipeline_experiment"), run_id="run_id")
published_pipeline = pipeline_run.publish_pipeline(name="My_New_Pipeline",
description="My New Pipeline Description",
version="1.0",
continue_on_step_failure=True)
Note: the continue_on_step_failure parameter specifies whether the execution of steps in the Pipeline will continue if one step fails. The default value is False, meaning when one step fails, the Pipeline execution will stop, canceling any running steps.
Submit a PublishedPipeline using submit. When submit is called, a PipelineRun is created which in turn creates StepRun objects for each step in the workflow.
An example to submit a PublishedPipeline is as follows:
from azureml.pipeline.core import PublishedPipeline
published_pipeline = PublishedPipeline.get(workspace=ws, id="published_pipeline_id")
pipeline_run = experiment.submit(published_pipeline)
There are a number of optional settings that can be specified when submitting a PublishedPipeline. These include:
continue_on_step_failure: Whether to continue execution of other steps in the PipelineRun if a step fails, optional. Only steps that have no dependency on the output of the failed step will continue execution. If provided, this parameter setting overrides the setting on the Pipeline.
pipeline_parameters: Parameters to pipeline execution, dictionary of {name: value}. See PipelineParameter for more details.
parent_run_id: You can supply a run ID to set the parent run of this pipeline run, which is reflected in RunHistory. The parent run must belong to the same experiment as the pipeline being submitted.
An example to submit a PublishedPipeline using these settings is as follows:
from azureml.pipeline.core import PublishedPipeline
published_pipeline = PublishedPipeline.get(workspace=ws, id="published_pipeline_id")
pipeline_run = experiment.submit(published_pipeline,
continue_on_step_failure=True,
pipeline_parameters={"param1": "value1"},
parent_run_id="<run_id>")
All published pipelines have a REST endpoint. With the pipeline endpoint, you can trigger a run of the pipeline from external systems, such as non-Python clients. For information about how to authenticate when calling REST endpoints, see https://aka.ms/pl-restep-auth.
Using the endpoint enables "managed repeatability" in batch scoring and retraining scenarios, for example. For more information, see https://aka.ms/pl-first-pipeline.
Methods
disable |
Set the published pipeline to 'Disabled' and unavailable to run. |
enable |
Set the published pipeline to 'Active' and available to run. |
get |
Get the published pipeline. |
get_all |
Get all published pipelines in the current workspace. DEPRECATED: This method is being deprecated in favor of PublishedPipeline list method. |
get_graph |
Get the graph of the PublishedPipeline. |
get_step_names |
Get the list of names of steps in the PublishedPipeline. |
list |
Get all published pipelines in the current workspace. |
save |
Save the Pipeline YAML to a file. Currently, only pipelines consisting of ModuleSteps are supported for YAML exporting. |
submit |
Submit the published pipeline. This is equivalent to using submit. Returns the submitted PipelineRun. Use this object to monitor and view details of the run. |
disable
Set the published pipeline to 'Disabled' and unavailable to run.
disable()
enable
Set the published pipeline to 'Active' and available to run.
enable()
get
Get the published pipeline.
static get(workspace, id, _workflow_provider=None, _service_endpoint=None)
Parameters
Name | Description |
---|---|
workspace
Required
|
The workspace the published pipeline was created in. |
id
Required
|
The ID of the published pipeline. |
_workflow_provider
|
<xref:azureml.pipeline.core._aeva_provider._AevaWorkflowProvider>
The workflow provider. Default value: None
|
_service_endpoint
|
The service endpoint. Default value: None
|
Returns
Type | Description |
---|---|
A PublishedPipeline object. |
get_all
Get all published pipelines in the current workspace.
DEPRECATED: This method is being deprecated in favor of PublishedPipeline list method.
static get_all(workspace, active_only=True, _service_endpoint=None)
Parameters
Name | Description |
---|---|
workspace
Required
|
The workspace the published pipeline was created on. |
active_only
|
Whether to return only published pipelines which are currently active. Default value: True
|
_service_endpoint
|
The service endpoint. Default value: None
|
Returns
Type | Description |
---|---|
A list of PublishedPipeline objects. |
get_graph
Get the graph of the PublishedPipeline.
get_graph(_workflow_provider=None)
Parameters
Name | Description |
---|---|
_workflow_provider
|
<xref:azureml.pipeline.core._aeva_provider._AevaWorkflowProvider>
The workflow provider. Default value: None
|
Returns
Type | Description |
---|---|
The graph. |
get_step_names
Get the list of names of steps in the PublishedPipeline.
get_step_names(_workflow_provider=None)
Parameters
Name | Description |
---|---|
_workflow_provider
|
<xref:azureml.pipeline.core._aeva_provider._AevaWorkflowProvider>
The workflow provider. Default value: None
|
Returns
Type | Description |
---|---|
The list of the names of steps in the PublishedPipeline. |
list
Get all published pipelines in the current workspace.
static list(workspace, active_only=True, _service_endpoint=None)
Parameters
Name | Description |
---|---|
workspace
Required
|
The workspace the published pipeline was created in. |
active_only
|
Whether to return only published pipelines which are currently active. Default value: True
|
_service_endpoint
|
The service endpoint. Default value: None
|
Returns
Type | Description |
---|---|
A list of PublishedPipeline objects. |
save
Save the Pipeline YAML to a file.
Currently, only pipelines consisting of ModuleSteps are supported for YAML exporting.
save(path=None, _workflow_provider=None)
Parameters
Name | Description |
---|---|
path
|
The path to save the YAML to. If the path is a directory, the Pipeline YAML file is saved at path/pipeline_name.yml. If path is None, the current directory is used. Default value: None
|
_workflow_provider
|
<xref:azureml.pipeline.core._aeva_provider._AevaWorkflowProvider>
The workflow provider. Default value: None
|
Returns
Type | Description |
---|---|
submit
Submit the published pipeline. This is equivalent to using submit.
Returns the submitted PipelineRun. Use this object to monitor and view details of the run.
submit(workspace, experiment_name, pipeline_parameters=None, _workflow_provider=None, _service_endpoint=None, parent_run_id=None, continue_on_step_failure=None)
Parameters
Name | Description |
---|---|
workspace
Required
|
The workspace to submit the published pipeline on. |
experiment_name
Required
|
The name of the experiment to submit to. |
pipeline_parameters
|
A dictionary of parameters to assign new values {param name, param value}. See PipelineParameter for more details. Default value: None
|
_workflow_provider
|
<xref:azureml.pipeline.core._aeva_provider._AevaWorkflowProvider>
The workflow provider. Default value: None
|
_service_endpoint
|
The service endpoint. Default value: None
|
parent_run_id
|
Optional run ID to set for the parent run of this pipeline run, which is reflected in RunHistory. The parent run must belong to same experiment as this pipeline is being submitted to. Default value: None
|
continue_on_step_failure
|
Whether to continue execution of other steps in the PipelineRun if a step fails, optional. If provided, will override the setting on the Pipeline. Default value: None
|
Returns
Type | Description |
---|---|
The submitted pipeline run. |
Attributes
continue_on_step_failure
Get the value of the continue_on_step_failure
setting.
Returns
Type | Description |
---|---|
The value of the |
description
Get the description of the published pipeline.
Returns
Type | Description |
---|---|
The description of the published pipeline. |
endpoint
Get the REST endpoint URL for running a published pipeline.
Returns
Type | Description |
---|---|
The REST endpoint URL for running the published pipeline. |
graph_id
Get the ID of the graph for this published pipeline.
Returns
Type | Description |
---|---|
The ID of the graph. |
id
name
status
Get the status of the published pipeline.
Returns
Type | Description |
---|---|
The status of the published pipeline. |
total_run_steps
Get the number of steps in the pipeline.
Returns
Type | Description |
---|---|
The number of steps in the pipeline. |
version
Get the version of the published pipeline.
Returns
Type | Description |
---|---|
The version of the published pipeline. |