Microsoft.MachineLearningServices workspaces/jobs 2021-03-01-preview

Bicep resource definition

The workspaces/jobs resource type can be deployed with operations that target:

For a list of changed properties in each API version, see change log.

Resource format

To create a Microsoft.MachineLearningServices/workspaces/jobs resource, add the following Bicep to your template.

resource symbolicname 'Microsoft.MachineLearningServices/workspaces/jobs@2021-03-01-preview' = {
  parent: resourceSymbolicName
  name: 'string'
  properties: {
    description: 'string'
    properties: {
      {customized property}: 'string'
    }
    tags: {
      {customized property}: 'string'
    }
    jobType: 'string'
    // For remaining properties, see JobBase objects
  }
}

EarlyTerminationPolicy objects

Set the policyType property to specify the type of object.

For Bandit, use:

{
  policyType: 'Bandit'
  slackAmount: int
  slackFactor: int
}

For MedianStopping, use:

{
  policyType: 'MedianStopping'
}

For TruncationSelection, use:

{
  policyType: 'TruncationSelection'
  truncationPercentage: int
}

IdentityConfiguration objects

Set the identityType property to specify the type of object.

For AMLToken, use:

{
  identityType: 'AMLToken'
}

For Managed, use:

{
  clientId: 'string'
  identityType: 'Managed'
  objectId: 'string'
  resourceId: 'string'
}

JobBase objects

Set the jobType property to specify the type of object.

For Command, use:

{
  codeId: 'string'
  command: 'string'
  compute: {
    instanceCount: int
    instanceType: 'string'
    isLocal: bool
    location: 'string'
    properties: {
      {customized property}: 'string'
    }
    target: 'string'
  }
  distribution: {
    distributionType: 'string'
    // For remaining properties, see DistributionConfiguration objects
  }
  environmentId: 'string'
  environmentVariables: {
    {customized property}: 'string'
  }
  experimentName: 'string'
  identity: {
    identityType: 'string'
    // For remaining properties, see IdentityConfiguration objects
  }
  inputDataBindings: {
    {customized property}: {
      dataId: 'string'
      mode: 'string'
      pathOnCompute: 'string'
    }
  }
  jobType: 'Command'
  outputDataBindings: {
    {customized property}: {
      datastoreId: 'string'
      mode: 'string'
      pathOnCompute: 'string'
      pathOnDatastore: 'string'
    }
  }
  priority: int
  timeout: 'string'
}

For Sweep, use:

{
  algorithm: 'string'
  compute: {
    instanceCount: int
    instanceType: 'string'
    isLocal: bool
    location: 'string'
    properties: {
      {customized property}: 'string'
    }
    target: 'string'
  }
  earlyTermination: {
    delayEvaluation: int
    evaluationInterval: int
    policyType: 'string'
    // For remaining properties, see EarlyTerminationPolicy objects
  }
  experimentName: 'string'
  identity: {
    identityType: 'string'
    // For remaining properties, see IdentityConfiguration objects
  }
  jobType: 'Sweep'
  maxConcurrentTrials: int
  maxTotalTrials: int
  objective: {
    goal: 'string'
    primaryMetric: 'string'
  }
  priority: int
  searchSpace: {
    {customized property}: any(Azure.Bicep.Types.Concrete.AnyType)
  }
  timeout: 'string'
  trial: {
    codeId: 'string'
    command: 'string'
    distribution: {
      distributionType: 'string'
      // For remaining properties, see DistributionConfiguration objects
    }
    environmentId: 'string'
    environmentVariables: {
      {customized property}: 'string'
    }
    inputDataBindings: {
      {customized property}: {
        dataId: 'string'
        mode: 'string'
        pathOnCompute: 'string'
      }
    }
    outputDataBindings: {
      {customized property}: {
        datastoreId: 'string'
        mode: 'string'
        pathOnCompute: 'string'
        pathOnDatastore: 'string'
      }
    }
    timeout: 'string'
  }
}

DistributionConfiguration objects

Set the distributionType property to specify the type of object.

For Mpi, use:

{
  distributionType: 'Mpi'
  processCountPerInstance: int
}

For PyTorch, use:

{
  distributionType: 'PyTorch'
  processCount: int
}

For TensorFlow, use:

{
  distributionType: 'TensorFlow'
  parameterServerCount: int
  workerCount: int
}

Property values

AmlToken

Name Description Value
identityType [Required] Specifies the type of identity framework. 'AMLToken' (required)

BanditPolicy

Name Description Value
policyType [Required] Name of policy configuration 'Bandit' (required)
slackAmount Absolute distance allowed from the best performing run. int
slackFactor Ratio of the allowed distance from the best performing run. int

CommandJob

Name Description Value
codeId ARM resource ID of the code asset. string
command [Required] The command to execute on startup of the job. eg. "python train.py" string

Constraints:
Min length = 1
Pattern = [a-zA-Z0-9_] (required)
compute [Required] Compute binding for the job. ComputeConfiguration (required)
distribution Distribution configuration of the job. If set, this should be one of Mpi, Tensorflow, PyTorch, or null. DistributionConfiguration
environmentId The ARM resource ID of the Environment specification for the job. string
environmentVariables Environment variables included in the job. CommandJobEnvironmentVariables
experimentName The name of the experiment the job belongs to. If not set, the job is placed in the "Default" experiment. string
identity Identity configuration. If set, this should be one of AmlToken, ManagedIdentity, or null.
Defaults to AmlToken if null.
IdentityConfiguration
inputDataBindings Mapping of input data bindings used in the job. CommandJobInputDataBindings
jobType [Required] Specifies the type of job. 'Command' (required)
outputDataBindings Mapping of output data bindings used in the job. CommandJobOutputDataBindings
priority Job priority for scheduling policy. Only applies to AMLCompute.
Private preview feature and only available to users on the allow list.
int
timeout The max run duration in ISO 8601 format, after which the job will be cancelled. Only supports duration with precision as low as Seconds. string

CommandJobEnvironmentVariables

Name Description Value

CommandJobInputDataBindings

Name Description Value

CommandJobOutputDataBindings

Name Description Value

ComputeConfiguration

Name Description Value
instanceCount Number of instances or nodes. int
instanceType SKU type to run on. string
isLocal Set to true for jobs running on local compute. bool
location Location for virtual cluster run. string
properties Additional properties. ComputeConfigurationProperties
target ARM resource ID of the Compute you are targeting. If not provided the resource will be deployed as Managed. string

ComputeConfigurationProperties

Name Description Value

DistributionConfiguration

Name Description Value
distributionType Set to 'Mpi' for type Mpi. Set to 'PyTorch' for type PyTorch. Set to 'TensorFlow' for type TensorFlow. 'Mpi'
'PyTorch'
'TensorFlow' (required)

EarlyTerminationPolicy

Name Description Value
delayEvaluation Number of intervals by which to delay the first evaluation. int
evaluationInterval Interval (number of runs) between policy evaluations. int
policyType Set to 'Bandit' for type BanditPolicy. Set to 'MedianStopping' for type MedianStoppingPolicy. Set to 'TruncationSelection' for type TruncationSelectionPolicy. 'Bandit'
'MedianStopping'
'TruncationSelection' (required)

IdentityConfiguration

Name Description Value
identityType Set to 'AMLToken' for type AmlToken. Set to 'Managed' for type ManagedIdentity. 'AMLToken'
'Managed' (required)

InputDataBinding

Name Description Value
dataId ARM resource ID of the registered dataVersion. string
mode Mechanism for accessing the data artifact. 'Direct'
'Download'
'EvalDownload'
'EvalMount'
'Mount'
'ReadOnlyMount'
'ReadWriteMount'
'Upload'
pathOnCompute Location of data inside the container process. string

JobBase

Name Description Value
description The asset description text. string
jobType Set to 'Command' for type CommandJob. Set to 'Sweep' for type SweepJob. 'Command'
'Sweep' (required)
properties The asset property dictionary. JobBaseProperties
tags Tag dictionary. Tags can be added, removed, and updated. JobBaseTags

JobBaseProperties

Name Description Value

JobBaseTags

Name Description Value

ManagedIdentity

Name Description Value
clientId Specifies a user-assigned identity by client ID. For system-assigned, do not set this field. string

Constraints:
Min length = 36
Max length = 36
Pattern = ^[0-9a-fA-F]{8}-([0-9a-fA-F]{4}-){3}[0-9a-fA-F]{12}$
identityType [Required] Specifies the type of identity framework. 'Managed' (required)
objectId Specifies a user-assigned identity by object ID. For system-assigned, do not set this field. string

Constraints:
Min length = 36
Max length = 36
Pattern = ^[0-9a-fA-F]{8}-([0-9a-fA-F]{4}-){3}[0-9a-fA-F]{12}$
resourceId Specifies a user-assigned identity by ARM resource ID. For system-assigned, do not set this field. string

MedianStoppingPolicy

Name Description Value
policyType [Required] Name of policy configuration 'MedianStopping' (required)

Microsoft.MachineLearningServices/workspaces/jobs

Name Description Value
name The resource name string

Constraints:
Pattern = ^[a-zA-Z0-9][a-zA-Z0-9\-_]{0,254}$ (required)
parent In Bicep, you can specify the parent resource for a child resource. You only need to add this property when the child resource is declared outside of the parent resource.

For more information, see Child resource outside parent resource.
Symbolic name for resource of type: workspaces
properties [Required] Additional attributes of the entity. JobBase (required)

Mpi

Name Description Value
distributionType [Required] Specifies the type of distribution framework. 'Mpi' (required)
processCountPerInstance Number of processes per MPI node. int

Objective

Name Description Value
goal [Required] Defines supported metric goals for hyperparameter tuning 'Maximize'
'Minimize' (required)
primaryMetric [Required] Name of the metric to optimize. string

Constraints:
Pattern = [a-zA-Z0-9_] (required)

OutputDataBinding

Name Description Value
datastoreId ARM resource ID of the datastore where the data output will be stored. string
mode Mechanism for data movement to datastore. 'Direct'
'Download'
'EvalDownload'
'EvalMount'
'Mount'
'ReadOnlyMount'
'ReadWriteMount'
'Upload'
pathOnCompute Location of data inside the container process. string
pathOnDatastore Path within the datastore to the data. string

PyTorch

Name Description Value
distributionType [Required] Specifies the type of distribution framework. 'PyTorch' (required)
processCount Total process count for the distributed job. int

SweepJob

Name Description Value
algorithm [Required] Type of the hyperparameter sampling algorithms 'Bayesian'
'Grid'
'Random' (required)
compute [Required] Compute binding for the job. ComputeConfiguration (required)
earlyTermination Early termination policies enable canceling poor-performing runs before they complete. EarlyTerminationPolicy
experimentName The name of the experiment the job belongs to. If not set, the job is placed in the "Default" experiment. string
identity Identity configuration. If set, this should be one of AmlToken, ManagedIdentity or null.
Defaults to AmlToken if null.
IdentityConfiguration
jobType [Required] Specifies the type of job. 'Sweep' (required)
maxConcurrentTrials An upper bound on the number of trials performed in parallel. int
maxTotalTrials An upper bound on the number of trials to perform. int
objective [Required] Optimization objective. Objective (required)
priority Job priority for scheduling policy. Only applies to AMLCompute.
Private preview feature and only available to users on the allow list.
int
searchSpace [Required] A dictionary containing each parameter and its distribution. The dictionary key is the name of the parameter SweepJobSearchSpace (required)
timeout The total timeout in ISO 8601 format. Only supports duration with precision as low as Minutes. string
trial Trial component definition. TrialComponent

SweepJobSearchSpace

Name Description Value

TensorFlow

Name Description Value
distributionType [Required] Specifies the type of distribution framework. 'TensorFlow' (required)
parameterServerCount Number of parameter server tasks. int
workerCount Number of workers. Overwrites the node count in compute binding. int

TrialComponent

Name Description Value
codeId ARM resource ID of the code asset. string
command [Required] The command to execute on startup of the job. eg. "python train.py" string

Constraints:
Min length = 1
Pattern = [a-zA-Z0-9_] (required)
distribution Distribution configuration of the job. If set, this should be one of Mpi, Tensorflow, PyTorch, or null. DistributionConfiguration
environmentId The ARM resource ID of the Environment specification for the job. string
environmentVariables Environment variables included in the job. TrialComponentEnvironmentVariables
inputDataBindings Mapping of input data bindings used in the job. TrialComponentInputDataBindings
outputDataBindings Mapping of output data bindings used in the job. TrialComponentOutputDataBindings
timeout The max run duration in ISO 8601 format, after which the trial component will be cancelled.
Only supports duration with precision as low as Seconds.
string

TrialComponentEnvironmentVariables

Name Description Value

TrialComponentInputDataBindings

Name Description Value

TrialComponentOutputDataBindings

Name Description Value

TruncationSelectionPolicy

Name Description Value
policyType [Required] Name of policy configuration 'TruncationSelection' (required)
truncationPercentage The percentage of runs to cancel at each evaluation interval. int

Quickstart samples

The following quickstart samples deploy this resource type.

Bicep File Description
Create an Azure Machine Learning AutoML classification job This template creates an Azure Machine Learning AutoML classification job to find out the best model for predicting if a client will subscribe to a fixed term deposit with a financial institution.
Create an Azure Machine Learning Command job This template creates an Azure Machine Learning Command job with a basic hello_world script
Create an Azure Machine Learning Sweep job This template creates an Azure Machine Learning Sweep job for hyperparameter tuning.

ARM template resource definition

The workspaces/jobs resource type can be deployed with operations that target:

For a list of changed properties in each API version, see change log.

Resource format

To create a Microsoft.MachineLearningServices/workspaces/jobs resource, add the following JSON to your template.

{
  "type": "Microsoft.MachineLearningServices/workspaces/jobs",
  "apiVersion": "2021-03-01-preview",
  "name": "string",
  "properties": {
    "description": "string",
    "properties": {
      "{customized property}": "string"
    },
    "tags": {
      "{customized property}": "string"
    },
    "jobType": "string"
    // For remaining properties, see JobBase objects
  }
}

EarlyTerminationPolicy objects

Set the policyType property to specify the type of object.

For Bandit, use:

{
  "policyType": "Bandit",
  "slackAmount": "int",
  "slackFactor": "int"
}

For MedianStopping, use:

{
  "policyType": "MedianStopping"
}

For TruncationSelection, use:

{
  "policyType": "TruncationSelection",
  "truncationPercentage": "int"
}

IdentityConfiguration objects

Set the identityType property to specify the type of object.

For AMLToken, use:

{
  "identityType": "AMLToken"
}

For Managed, use:

{
  "clientId": "string",
  "identityType": "Managed",
  "objectId": "string",
  "resourceId": "string"
}

JobBase objects

Set the jobType property to specify the type of object.

For Command, use:

{
  "codeId": "string",
  "command": "string",
  "compute": {
    "instanceCount": "int",
    "instanceType": "string",
    "isLocal": "bool",
    "location": "string",
    "properties": {
      "{customized property}": "string"
    },
    "target": "string"
  },
  "distribution": {
    "distributionType": "string"
    // For remaining properties, see DistributionConfiguration objects
  },
  "environmentId": "string",
  "environmentVariables": {
    "{customized property}": "string"
  },
  "experimentName": "string",
  "identity": {
    "identityType": "string"
    // For remaining properties, see IdentityConfiguration objects
  },
  "inputDataBindings": {
    "{customized property}": {
      "dataId": "string",
      "mode": "string",
      "pathOnCompute": "string"
    }
  },
  "jobType": "Command",
  "outputDataBindings": {
    "{customized property}": {
      "datastoreId": "string",
      "mode": "string",
      "pathOnCompute": "string",
      "pathOnDatastore": "string"
    }
  },
  "priority": "int",
  "timeout": "string"
}

For Sweep, use:

{
  "algorithm": "string",
  "compute": {
    "instanceCount": "int",
    "instanceType": "string",
    "isLocal": "bool",
    "location": "string",
    "properties": {
      "{customized property}": "string"
    },
    "target": "string"
  },
  "earlyTermination": {
    "delayEvaluation": "int",
    "evaluationInterval": "int",
    "policyType": "string"
    // For remaining properties, see EarlyTerminationPolicy objects
  },
  "experimentName": "string",
  "identity": {
    "identityType": "string"
    // For remaining properties, see IdentityConfiguration objects
  },
  "jobType": "Sweep",
  "maxConcurrentTrials": "int",
  "maxTotalTrials": "int",
  "objective": {
    "goal": "string",
    "primaryMetric": "string"
  },
  "priority": "int",
  "searchSpace": {
    "{customized property}": {}
  },
  "timeout": "string",
  "trial": {
    "codeId": "string",
    "command": "string",
    "distribution": {
      "distributionType": "string"
      // For remaining properties, see DistributionConfiguration objects
    },
    "environmentId": "string",
    "environmentVariables": {
      "{customized property}": "string"
    },
    "inputDataBindings": {
      "{customized property}": {
        "dataId": "string",
        "mode": "string",
        "pathOnCompute": "string"
      }
    },
    "outputDataBindings": {
      "{customized property}": {
        "datastoreId": "string",
        "mode": "string",
        "pathOnCompute": "string",
        "pathOnDatastore": "string"
      }
    },
    "timeout": "string"
  }
}

DistributionConfiguration objects

Set the distributionType property to specify the type of object.

For Mpi, use:

{
  "distributionType": "Mpi",
  "processCountPerInstance": "int"
}

For PyTorch, use:

{
  "distributionType": "PyTorch",
  "processCount": "int"
}

For TensorFlow, use:

{
  "distributionType": "TensorFlow",
  "parameterServerCount": "int",
  "workerCount": "int"
}

Property values

AmlToken

Name Description Value
identityType [Required] Specifies the type of identity framework. 'AMLToken' (required)

BanditPolicy

Name Description Value
policyType [Required] Name of policy configuration 'Bandit' (required)
slackAmount Absolute distance allowed from the best performing run. int
slackFactor Ratio of the allowed distance from the best performing run. int

CommandJob

Name Description Value
codeId ARM resource ID of the code asset. string
command [Required] The command to execute on startup of the job. eg. "python train.py" string

Constraints:
Min length = 1
Pattern = [a-zA-Z0-9_] (required)
compute [Required] Compute binding for the job. ComputeConfiguration (required)
distribution Distribution configuration of the job. If set, this should be one of Mpi, Tensorflow, PyTorch, or null. DistributionConfiguration
environmentId The ARM resource ID of the Environment specification for the job. string
environmentVariables Environment variables included in the job. CommandJobEnvironmentVariables
experimentName The name of the experiment the job belongs to. If not set, the job is placed in the "Default" experiment. string
identity Identity configuration. If set, this should be one of AmlToken, ManagedIdentity, or null.
Defaults to AmlToken if null.
IdentityConfiguration
inputDataBindings Mapping of input data bindings used in the job. CommandJobInputDataBindings
jobType [Required] Specifies the type of job. 'Command' (required)
outputDataBindings Mapping of output data bindings used in the job. CommandJobOutputDataBindings
priority Job priority for scheduling policy. Only applies to AMLCompute.
Private preview feature and only available to users on the allow list.
int
timeout The max run duration in ISO 8601 format, after which the job will be cancelled. Only supports duration with precision as low as Seconds. string

CommandJobEnvironmentVariables

Name Description Value

CommandJobInputDataBindings

Name Description Value

CommandJobOutputDataBindings

Name Description Value

ComputeConfiguration

Name Description Value
instanceCount Number of instances or nodes. int
instanceType SKU type to run on. string
isLocal Set to true for jobs running on local compute. bool
location Location for virtual cluster run. string
properties Additional properties. ComputeConfigurationProperties
target ARM resource ID of the Compute you are targeting. If not provided the resource will be deployed as Managed. string

ComputeConfigurationProperties

Name Description Value

DistributionConfiguration

Name Description Value
distributionType Set to 'Mpi' for type Mpi. Set to 'PyTorch' for type PyTorch. Set to 'TensorFlow' for type TensorFlow. 'Mpi'
'PyTorch'
'TensorFlow' (required)

EarlyTerminationPolicy

Name Description Value
delayEvaluation Number of intervals by which to delay the first evaluation. int
evaluationInterval Interval (number of runs) between policy evaluations. int
policyType Set to 'Bandit' for type BanditPolicy. Set to 'MedianStopping' for type MedianStoppingPolicy. Set to 'TruncationSelection' for type TruncationSelectionPolicy. 'Bandit'
'MedianStopping'
'TruncationSelection' (required)

IdentityConfiguration

Name Description Value
identityType Set to 'AMLToken' for type AmlToken. Set to 'Managed' for type ManagedIdentity. 'AMLToken'
'Managed' (required)

InputDataBinding

Name Description Value
dataId ARM resource ID of the registered dataVersion. string
mode Mechanism for accessing the data artifact. 'Direct'
'Download'
'EvalDownload'
'EvalMount'
'Mount'
'ReadOnlyMount'
'ReadWriteMount'
'Upload'
pathOnCompute Location of data inside the container process. string

JobBase

Name Description Value
description The asset description text. string
jobType Set to 'Command' for type CommandJob. Set to 'Sweep' for type SweepJob. 'Command'
'Sweep' (required)
properties The asset property dictionary. JobBaseProperties
tags Tag dictionary. Tags can be added, removed, and updated. JobBaseTags

JobBaseProperties

Name Description Value

JobBaseTags

Name Description Value

ManagedIdentity

Name Description Value
clientId Specifies a user-assigned identity by client ID. For system-assigned, do not set this field. string

Constraints:
Min length = 36
Max length = 36
Pattern = ^[0-9a-fA-F]{8}-([0-9a-fA-F]{4}-){3}[0-9a-fA-F]{12}$
identityType [Required] Specifies the type of identity framework. 'Managed' (required)
objectId Specifies a user-assigned identity by object ID. For system-assigned, do not set this field. string

Constraints:
Min length = 36
Max length = 36
Pattern = ^[0-9a-fA-F]{8}-([0-9a-fA-F]{4}-){3}[0-9a-fA-F]{12}$
resourceId Specifies a user-assigned identity by ARM resource ID. For system-assigned, do not set this field. string

MedianStoppingPolicy

Name Description Value
policyType [Required] Name of policy configuration 'MedianStopping' (required)

Microsoft.MachineLearningServices/workspaces/jobs

Name Description Value
apiVersion The api version '2021-03-01-preview'
name The resource name string

Constraints:
Pattern = ^[a-zA-Z0-9][a-zA-Z0-9\-_]{0,254}$ (required)
properties [Required] Additional attributes of the entity. JobBase (required)
type The resource type 'Microsoft.MachineLearningServices/workspaces/jobs'

Mpi

Name Description Value
distributionType [Required] Specifies the type of distribution framework. 'Mpi' (required)
processCountPerInstance Number of processes per MPI node. int

Objective

Name Description Value
goal [Required] Defines supported metric goals for hyperparameter tuning 'Maximize'
'Minimize' (required)
primaryMetric [Required] Name of the metric to optimize. string

Constraints:
Pattern = [a-zA-Z0-9_] (required)

OutputDataBinding

Name Description Value
datastoreId ARM resource ID of the datastore where the data output will be stored. string
mode Mechanism for data movement to datastore. 'Direct'
'Download'
'EvalDownload'
'EvalMount'
'Mount'
'ReadOnlyMount'
'ReadWriteMount'
'Upload'
pathOnCompute Location of data inside the container process. string
pathOnDatastore Path within the datastore to the data. string

PyTorch

Name Description Value
distributionType [Required] Specifies the type of distribution framework. 'PyTorch' (required)
processCount Total process count for the distributed job. int

SweepJob

Name Description Value
algorithm [Required] Type of the hyperparameter sampling algorithms 'Bayesian'
'Grid'
'Random' (required)
compute [Required] Compute binding for the job. ComputeConfiguration (required)
earlyTermination Early termination policies enable canceling poor-performing runs before they complete. EarlyTerminationPolicy
experimentName The name of the experiment the job belongs to. If not set, the job is placed in the "Default" experiment. string
identity Identity configuration. If set, this should be one of AmlToken, ManagedIdentity or null.
Defaults to AmlToken if null.
IdentityConfiguration
jobType [Required] Specifies the type of job. 'Sweep' (required)
maxConcurrentTrials An upper bound on the number of trials performed in parallel. int
maxTotalTrials An upper bound on the number of trials to perform. int
objective [Required] Optimization objective. Objective (required)
priority Job priority for scheduling policy. Only applies to AMLCompute.
Private preview feature and only available to users on the allow list.
int
searchSpace [Required] A dictionary containing each parameter and its distribution. The dictionary key is the name of the parameter SweepJobSearchSpace (required)
timeout The total timeout in ISO 8601 format. Only supports duration with precision as low as Minutes. string
trial Trial component definition. TrialComponent

SweepJobSearchSpace

Name Description Value

TensorFlow

Name Description Value
distributionType [Required] Specifies the type of distribution framework. 'TensorFlow' (required)
parameterServerCount Number of parameter server tasks. int
workerCount Number of workers. Overwrites the node count in compute binding. int

TrialComponent

Name Description Value
codeId ARM resource ID of the code asset. string
command [Required] The command to execute on startup of the job. eg. "python train.py" string

Constraints:
Min length = 1
Pattern = [a-zA-Z0-9_] (required)
distribution Distribution configuration of the job. If set, this should be one of Mpi, Tensorflow, PyTorch, or null. DistributionConfiguration
environmentId The ARM resource ID of the Environment specification for the job. string
environmentVariables Environment variables included in the job. TrialComponentEnvironmentVariables
inputDataBindings Mapping of input data bindings used in the job. TrialComponentInputDataBindings
outputDataBindings Mapping of output data bindings used in the job. TrialComponentOutputDataBindings
timeout The max run duration in ISO 8601 format, after which the trial component will be cancelled.
Only supports duration with precision as low as Seconds.
string

TrialComponentEnvironmentVariables

Name Description Value

TrialComponentInputDataBindings

Name Description Value

TrialComponentOutputDataBindings

Name Description Value

TruncationSelectionPolicy

Name Description Value
policyType [Required] Name of policy configuration 'TruncationSelection' (required)
truncationPercentage The percentage of runs to cancel at each evaluation interval. int

Quickstart templates

The following quickstart templates deploy this resource type.

Template Description
Create an Azure Machine Learning AutoML classification job

Deploy to Azure
This template creates an Azure Machine Learning AutoML classification job to find out the best model for predicting if a client will subscribe to a fixed term deposit with a financial institution.
Create an Azure Machine Learning Command job

Deploy to Azure
This template creates an Azure Machine Learning Command job with a basic hello_world script
Create an Azure Machine Learning Sweep job

Deploy to Azure
This template creates an Azure Machine Learning Sweep job for hyperparameter tuning.

Terraform (AzAPI provider) resource definition

The workspaces/jobs resource type can be deployed with operations that target:

  • Resource groups

For a list of changed properties in each API version, see change log.

Resource format

To create a Microsoft.MachineLearningServices/workspaces/jobs resource, add the following Terraform to your template.

resource "azapi_resource" "symbolicname" {
  type = "Microsoft.MachineLearningServices/workspaces/jobs@2021-03-01-preview"
  name = "string"
  body = jsonencode({
    properties = {
      description = "string"
      properties = {
        {customized property} = "string"
      }
      tags = {
        {customized property} = "string"
      }
      jobType = "string"
      // For remaining properties, see JobBase objects
    }
  })
}

EarlyTerminationPolicy objects

Set the policyType property to specify the type of object.

For Bandit, use:

{
  policyType = "Bandit"
  slackAmount = int
  slackFactor = int
}

For MedianStopping, use:

{
  policyType = "MedianStopping"
}

For TruncationSelection, use:

{
  policyType = "TruncationSelection"
  truncationPercentage = int
}

IdentityConfiguration objects

Set the identityType property to specify the type of object.

For AMLToken, use:

{
  identityType = "AMLToken"
}

For Managed, use:

{
  clientId = "string"
  identityType = "Managed"
  objectId = "string"
  resourceId = "string"
}

JobBase objects

Set the jobType property to specify the type of object.

For Command, use:

{
  codeId = "string"
  command = "string"
  compute = {
    instanceCount = int
    instanceType = "string"
    isLocal = bool
    location = "string"
    properties = {
      {customized property} = "string"
    }
    target = "string"
  }
  distribution = {
    distributionType = "string"
    // For remaining properties, see DistributionConfiguration objects
  }
  environmentId = "string"
  environmentVariables = {
    {customized property} = "string"
  }
  experimentName = "string"
  identity = {
    identityType = "string"
    // For remaining properties, see IdentityConfiguration objects
  }
  inputDataBindings = {
    {customized property} = {
      dataId = "string"
      mode = "string"
      pathOnCompute = "string"
    }
  }
  jobType = "Command"
  outputDataBindings = {
    {customized property} = {
      datastoreId = "string"
      mode = "string"
      pathOnCompute = "string"
      pathOnDatastore = "string"
    }
  }
  priority = int
  timeout = "string"
}

For Sweep, use:

{
  algorithm = "string"
  compute = {
    instanceCount = int
    instanceType = "string"
    isLocal = bool
    location = "string"
    properties = {
      {customized property} = "string"
    }
    target = "string"
  }
  earlyTermination = {
    delayEvaluation = int
    evaluationInterval = int
    policyType = "string"
    // For remaining properties, see EarlyTerminationPolicy objects
  }
  experimentName = "string"
  identity = {
    identityType = "string"
    // For remaining properties, see IdentityConfiguration objects
  }
  jobType = "Sweep"
  maxConcurrentTrials = int
  maxTotalTrials = int
  objective = {
    goal = "string"
    primaryMetric = "string"
  }
  priority = int
  searchSpace = {
    {customized property} = ?
  }
  timeout = "string"
  trial = {
    codeId = "string"
    command = "string"
    distribution = {
      distributionType = "string"
      // For remaining properties, see DistributionConfiguration objects
    }
    environmentId = "string"
    environmentVariables = {
      {customized property} = "string"
    }
    inputDataBindings = {
      {customized property} = {
        dataId = "string"
        mode = "string"
        pathOnCompute = "string"
      }
    }
    outputDataBindings = {
      {customized property} = {
        datastoreId = "string"
        mode = "string"
        pathOnCompute = "string"
        pathOnDatastore = "string"
      }
    }
    timeout = "string"
  }
}

DistributionConfiguration objects

Set the distributionType property to specify the type of object.

For Mpi, use:

{
  distributionType = "Mpi"
  processCountPerInstance = int
}

For PyTorch, use:

{
  distributionType = "PyTorch"
  processCount = int
}

For TensorFlow, use:

{
  distributionType = "TensorFlow"
  parameterServerCount = int
  workerCount = int
}

Property values

AmlToken

Name Description Value
identityType [Required] Specifies the type of identity framework. 'AMLToken' (required)

BanditPolicy

Name Description Value
policyType [Required] Name of policy configuration 'Bandit' (required)
slackAmount Absolute distance allowed from the best performing run. int
slackFactor Ratio of the allowed distance from the best performing run. int

CommandJob

Name Description Value
codeId ARM resource ID of the code asset. string
command [Required] The command to execute on startup of the job. eg. "python train.py" string

Constraints:
Min length = 1
Pattern = [a-zA-Z0-9_] (required)
compute [Required] Compute binding for the job. ComputeConfiguration (required)
distribution Distribution configuration of the job. If set, this should be one of Mpi, Tensorflow, PyTorch, or null. DistributionConfiguration
environmentId The ARM resource ID of the Environment specification for the job. string
environmentVariables Environment variables included in the job. CommandJobEnvironmentVariables
experimentName The name of the experiment the job belongs to. If not set, the job is placed in the "Default" experiment. string
identity Identity configuration. If set, this should be one of AmlToken, ManagedIdentity, or null.
Defaults to AmlToken if null.
IdentityConfiguration
inputDataBindings Mapping of input data bindings used in the job. CommandJobInputDataBindings
jobType [Required] Specifies the type of job. 'Command' (required)
outputDataBindings Mapping of output data bindings used in the job. CommandJobOutputDataBindings
priority Job priority for scheduling policy. Only applies to AMLCompute.
Private preview feature and only available to users on the allow list.
int
timeout The max run duration in ISO 8601 format, after which the job will be cancelled. Only supports duration with precision as low as Seconds. string

CommandJobEnvironmentVariables

Name Description Value

CommandJobInputDataBindings

Name Description Value

CommandJobOutputDataBindings

Name Description Value

ComputeConfiguration

Name Description Value
instanceCount Number of instances or nodes. int
instanceType SKU type to run on. string
isLocal Set to true for jobs running on local compute. bool
location Location for virtual cluster run. string
properties Additional properties. ComputeConfigurationProperties
target ARM resource ID of the Compute you are targeting. If not provided the resource will be deployed as Managed. string

ComputeConfigurationProperties

Name Description Value

DistributionConfiguration

Name Description Value
distributionType Set to 'Mpi' for type Mpi. Set to 'PyTorch' for type PyTorch. Set to 'TensorFlow' for type TensorFlow. 'Mpi'
'PyTorch'
'TensorFlow' (required)

EarlyTerminationPolicy

Name Description Value
delayEvaluation Number of intervals by which to delay the first evaluation. int
evaluationInterval Interval (number of runs) between policy evaluations. int
policyType Set to 'Bandit' for type BanditPolicy. Set to 'MedianStopping' for type MedianStoppingPolicy. Set to 'TruncationSelection' for type TruncationSelectionPolicy. 'Bandit'
'MedianStopping'
'TruncationSelection' (required)

IdentityConfiguration

Name Description Value
identityType Set to 'AMLToken' for type AmlToken. Set to 'Managed' for type ManagedIdentity. 'AMLToken'
'Managed' (required)

InputDataBinding

Name Description Value
dataId ARM resource ID of the registered dataVersion. string
mode Mechanism for accessing the data artifact. 'Direct'
'Download'
'EvalDownload'
'EvalMount'
'Mount'
'ReadOnlyMount'
'ReadWriteMount'
'Upload'
pathOnCompute Location of data inside the container process. string

JobBase

Name Description Value
description The asset description text. string
jobType Set to 'Command' for type CommandJob. Set to 'Sweep' for type SweepJob. 'Command'
'Sweep' (required)
properties The asset property dictionary. JobBaseProperties
tags Tag dictionary. Tags can be added, removed, and updated. JobBaseTags

JobBaseProperties

Name Description Value

JobBaseTags

Name Description Value

ManagedIdentity

Name Description Value
clientId Specifies a user-assigned identity by client ID. For system-assigned, do not set this field. string

Constraints:
Min length = 36
Max length = 36
Pattern = ^[0-9a-fA-F]{8}-([0-9a-fA-F]{4}-){3}[0-9a-fA-F]{12}$
identityType [Required] Specifies the type of identity framework. 'Managed' (required)
objectId Specifies a user-assigned identity by object ID. For system-assigned, do not set this field. string

Constraints:
Min length = 36
Max length = 36
Pattern = ^[0-9a-fA-F]{8}-([0-9a-fA-F]{4}-){3}[0-9a-fA-F]{12}$
resourceId Specifies a user-assigned identity by ARM resource ID. For system-assigned, do not set this field. string

MedianStoppingPolicy

Name Description Value
policyType [Required] Name of policy configuration 'MedianStopping' (required)

Microsoft.MachineLearningServices/workspaces/jobs

Name Description Value
name The resource name string

Constraints:
Pattern = ^[a-zA-Z0-9][a-zA-Z0-9\-_]{0,254}$ (required)
parent_id The ID of the resource that is the parent for this resource. ID for resource of type: workspaces
properties [Required] Additional attributes of the entity. JobBase (required)
type The resource type "Microsoft.MachineLearningServices/workspaces/jobs@2021-03-01-preview"

Mpi

Name Description Value
distributionType [Required] Specifies the type of distribution framework. 'Mpi' (required)
processCountPerInstance Number of processes per MPI node. int

Objective

Name Description Value
goal [Required] Defines supported metric goals for hyperparameter tuning 'Maximize'
'Minimize' (required)
primaryMetric [Required] Name of the metric to optimize. string

Constraints:
Pattern = [a-zA-Z0-9_] (required)

OutputDataBinding

Name Description Value
datastoreId ARM resource ID of the datastore where the data output will be stored. string
mode Mechanism for data movement to datastore. 'Direct'
'Download'
'EvalDownload'
'EvalMount'
'Mount'
'ReadOnlyMount'
'ReadWriteMount'
'Upload'
pathOnCompute Location of data inside the container process. string
pathOnDatastore Path within the datastore to the data. string

PyTorch

Name Description Value
distributionType [Required] Specifies the type of distribution framework. 'PyTorch' (required)
processCount Total process count for the distributed job. int

SweepJob

Name Description Value
algorithm [Required] Type of the hyperparameter sampling algorithms 'Bayesian'
'Grid'
'Random' (required)
compute [Required] Compute binding for the job. ComputeConfiguration (required)
earlyTermination Early termination policies enable canceling poor-performing runs before they complete. EarlyTerminationPolicy
experimentName The name of the experiment the job belongs to. If not set, the job is placed in the "Default" experiment. string
identity Identity configuration. If set, this should be one of AmlToken, ManagedIdentity or null.
Defaults to AmlToken if null.
IdentityConfiguration
jobType [Required] Specifies the type of job. 'Sweep' (required)
maxConcurrentTrials An upper bound on the number of trials performed in parallel. int
maxTotalTrials An upper bound on the number of trials to perform. int
objective [Required] Optimization objective. Objective (required)
priority Job priority for scheduling policy. Only applies to AMLCompute.
Private preview feature and only available to users on the allow list.
int
searchSpace [Required] A dictionary containing each parameter and its distribution. The dictionary key is the name of the parameter SweepJobSearchSpace (required)
timeout The total timeout in ISO 8601 format. Only supports duration with precision as low as Minutes. string
trial Trial component definition. TrialComponent

SweepJobSearchSpace

Name Description Value

TensorFlow

Name Description Value
distributionType [Required] Specifies the type of distribution framework. 'TensorFlow' (required)
parameterServerCount Number of parameter server tasks. int
workerCount Number of workers. Overwrites the node count in compute binding. int

TrialComponent

Name Description Value
codeId ARM resource ID of the code asset. string
command [Required] The command to execute on startup of the job. eg. "python train.py" string

Constraints:
Min length = 1
Pattern = [a-zA-Z0-9_] (required)
distribution Distribution configuration of the job. If set, this should be one of Mpi, Tensorflow, PyTorch, or null. DistributionConfiguration
environmentId The ARM resource ID of the Environment specification for the job. string
environmentVariables Environment variables included in the job. TrialComponentEnvironmentVariables
inputDataBindings Mapping of input data bindings used in the job. TrialComponentInputDataBindings
outputDataBindings Mapping of output data bindings used in the job. TrialComponentOutputDataBindings
timeout The max run duration in ISO 8601 format, after which the trial component will be cancelled.
Only supports duration with precision as low as Seconds.
string

TrialComponentEnvironmentVariables

Name Description Value

TrialComponentInputDataBindings

Name Description Value

TrialComponentOutputDataBindings

Name Description Value

TruncationSelectionPolicy

Name Description Value
policyType [Required] Name of policy configuration 'TruncationSelection' (required)
truncationPercentage The percentage of runs to cancel at each evaluation interval. int