Microsoft.MachineLearningServices workspaces/jobs 2021-03-01-preview
- Latest
- 2024-10-01
- 2024-10-01-preview
- 2024-07-01-preview
- 2024-04-01
- 2024-04-01-preview
- 2024-01-01-preview
- 2023-10-01
- 2023-08-01-preview
- 2023-06-01-preview
- 2023-04-01
- 2023-04-01-preview
- 2023-02-01-preview
- 2022-12-01-preview
- 2022-10-01
- 2022-10-01-preview
- 2022-06-01-preview
- 2022-05-01
- 2022-02-01-preview
- 2021-03-01-preview
Bicep resource definition
The workspaces/jobs resource type can be deployed with operations that target:
- Resource groups - See resource group deployment commands
For a list of changed properties in each API version, see change log.
Resource format
To create a Microsoft.MachineLearningServices/workspaces/jobs resource, add the following Bicep to your template.
resource symbolicname 'Microsoft.MachineLearningServices/workspaces/jobs@2021-03-01-preview' = {
parent: resourceSymbolicName
name: 'string'
properties: {
description: 'string'
properties: {
{customized property}: 'string'
}
tags: {
{customized property}: 'string'
}
jobType: 'string'
// For remaining properties, see JobBase objects
}
}
EarlyTerminationPolicy objects
Set the policyType property to specify the type of object.
For Bandit, use:
{
policyType: 'Bandit'
slackAmount: int
slackFactor: int
}
For MedianStopping, use:
{
policyType: 'MedianStopping'
}
For TruncationSelection, use:
{
policyType: 'TruncationSelection'
truncationPercentage: int
}
IdentityConfiguration objects
Set the identityType property to specify the type of object.
For AMLToken, use:
{
identityType: 'AMLToken'
}
For Managed, use:
{
clientId: 'string'
identityType: 'Managed'
objectId: 'string'
resourceId: 'string'
}
JobBase objects
Set the jobType property to specify the type of object.
For Command, use:
{
codeId: 'string'
command: 'string'
compute: {
instanceCount: int
instanceType: 'string'
isLocal: bool
location: 'string'
properties: {
{customized property}: 'string'
}
target: 'string'
}
distribution: {
distributionType: 'string'
// For remaining properties, see DistributionConfiguration objects
}
environmentId: 'string'
environmentVariables: {
{customized property}: 'string'
}
experimentName: 'string'
identity: {
identityType: 'string'
// For remaining properties, see IdentityConfiguration objects
}
inputDataBindings: {
{customized property}: {
dataId: 'string'
mode: 'string'
pathOnCompute: 'string'
}
}
jobType: 'Command'
outputDataBindings: {
{customized property}: {
datastoreId: 'string'
mode: 'string'
pathOnCompute: 'string'
pathOnDatastore: 'string'
}
}
priority: int
timeout: 'string'
}
For Sweep, use:
{
algorithm: 'string'
compute: {
instanceCount: int
instanceType: 'string'
isLocal: bool
location: 'string'
properties: {
{customized property}: 'string'
}
target: 'string'
}
earlyTermination: {
delayEvaluation: int
evaluationInterval: int
policyType: 'string'
// For remaining properties, see EarlyTerminationPolicy objects
}
experimentName: 'string'
identity: {
identityType: 'string'
// For remaining properties, see IdentityConfiguration objects
}
jobType: 'Sweep'
maxConcurrentTrials: int
maxTotalTrials: int
objective: {
goal: 'string'
primaryMetric: 'string'
}
priority: int
searchSpace: {
{customized property}: any(Azure.Bicep.Types.Concrete.AnyType)
}
timeout: 'string'
trial: {
codeId: 'string'
command: 'string'
distribution: {
distributionType: 'string'
// For remaining properties, see DistributionConfiguration objects
}
environmentId: 'string'
environmentVariables: {
{customized property}: 'string'
}
inputDataBindings: {
{customized property}: {
dataId: 'string'
mode: 'string'
pathOnCompute: 'string'
}
}
outputDataBindings: {
{customized property}: {
datastoreId: 'string'
mode: 'string'
pathOnCompute: 'string'
pathOnDatastore: 'string'
}
}
timeout: 'string'
}
}
DistributionConfiguration objects
Set the distributionType property to specify the type of object.
For Mpi, use:
{
distributionType: 'Mpi'
processCountPerInstance: int
}
For PyTorch, use:
{
distributionType: 'PyTorch'
processCount: int
}
For TensorFlow, use:
{
distributionType: 'TensorFlow'
parameterServerCount: int
workerCount: int
}
Property values
AmlToken
Name | Description | Value |
---|---|---|
identityType | [Required] Specifies the type of identity framework. | 'AMLToken' (required) |
BanditPolicy
Name | Description | Value |
---|---|---|
policyType | [Required] Name of policy configuration | 'Bandit' (required) |
slackAmount | Absolute distance allowed from the best performing run. | int |
slackFactor | Ratio of the allowed distance from the best performing run. | int |
CommandJob
Name | Description | Value |
---|---|---|
codeId | ARM resource ID of the code asset. | string |
command | [Required] The command to execute on startup of the job. eg. "python train.py" | string Constraints: Min length = 1 Pattern = [a-zA-Z0-9_] (required) |
compute | [Required] Compute binding for the job. | ComputeConfiguration (required) |
distribution | Distribution configuration of the job. If set, this should be one of Mpi, Tensorflow, PyTorch, or null. | DistributionConfiguration |
environmentId | The ARM resource ID of the Environment specification for the job. | string |
environmentVariables | Environment variables included in the job. | CommandJobEnvironmentVariables |
experimentName | The name of the experiment the job belongs to. If not set, the job is placed in the "Default" experiment. | string |
identity | Identity configuration. If set, this should be one of AmlToken, ManagedIdentity, or null. Defaults to AmlToken if null. |
IdentityConfiguration |
inputDataBindings | Mapping of input data bindings used in the job. | CommandJobInputDataBindings |
jobType | [Required] Specifies the type of job. | 'Command' (required) |
outputDataBindings | Mapping of output data bindings used in the job. | CommandJobOutputDataBindings |
priority | Job priority for scheduling policy. Only applies to AMLCompute. Private preview feature and only available to users on the allow list. |
int |
timeout | The max run duration in ISO 8601 format, after which the job will be cancelled. Only supports duration with precision as low as Seconds. | string |
CommandJobEnvironmentVariables
Name | Description | Value |
---|
CommandJobInputDataBindings
Name | Description | Value |
---|
CommandJobOutputDataBindings
Name | Description | Value |
---|
ComputeConfiguration
Name | Description | Value |
---|---|---|
instanceCount | Number of instances or nodes. | int |
instanceType | SKU type to run on. | string |
isLocal | Set to true for jobs running on local compute. | bool |
location | Location for virtual cluster run. | string |
properties | Additional properties. | ComputeConfigurationProperties |
target | ARM resource ID of the Compute you are targeting. If not provided the resource will be deployed as Managed. | string |
ComputeConfigurationProperties
Name | Description | Value |
---|
DistributionConfiguration
Name | Description | Value |
---|---|---|
distributionType | Set to 'Mpi' for type Mpi. Set to 'PyTorch' for type PyTorch. Set to 'TensorFlow' for type TensorFlow. | 'Mpi' 'PyTorch' 'TensorFlow' (required) |
EarlyTerminationPolicy
Name | Description | Value |
---|---|---|
delayEvaluation | Number of intervals by which to delay the first evaluation. | int |
evaluationInterval | Interval (number of runs) between policy evaluations. | int |
policyType | Set to 'Bandit' for type BanditPolicy. Set to 'MedianStopping' for type MedianStoppingPolicy. Set to 'TruncationSelection' for type TruncationSelectionPolicy. | 'Bandit' 'MedianStopping' 'TruncationSelection' (required) |
IdentityConfiguration
Name | Description | Value |
---|---|---|
identityType | Set to 'AMLToken' for type AmlToken. Set to 'Managed' for type ManagedIdentity. | 'AMLToken' 'Managed' (required) |
InputDataBinding
Name | Description | Value |
---|---|---|
dataId | ARM resource ID of the registered dataVersion. | string |
mode | Mechanism for accessing the data artifact. | 'Direct' 'Download' 'EvalDownload' 'EvalMount' 'Mount' 'ReadOnlyMount' 'ReadWriteMount' 'Upload' |
pathOnCompute | Location of data inside the container process. | string |
JobBase
Name | Description | Value |
---|---|---|
description | The asset description text. | string |
jobType | Set to 'Command' for type CommandJob. Set to 'Sweep' for type SweepJob. | 'Command' 'Sweep' (required) |
properties | The asset property dictionary. | JobBaseProperties |
tags | Tag dictionary. Tags can be added, removed, and updated. | JobBaseTags |
JobBaseProperties
Name | Description | Value |
---|
JobBaseTags
Name | Description | Value |
---|
ManagedIdentity
Name | Description | Value |
---|---|---|
clientId | Specifies a user-assigned identity by client ID. For system-assigned, do not set this field. | string Constraints: Min length = 36 Max length = 36 Pattern = ^[0-9a-fA-F]{8}-([0-9a-fA-F]{4}-){3}[0-9a-fA-F]{12}$ |
identityType | [Required] Specifies the type of identity framework. | 'Managed' (required) |
objectId | Specifies a user-assigned identity by object ID. For system-assigned, do not set this field. | string Constraints: Min length = 36 Max length = 36 Pattern = ^[0-9a-fA-F]{8}-([0-9a-fA-F]{4}-){3}[0-9a-fA-F]{12}$ |
resourceId | Specifies a user-assigned identity by ARM resource ID. For system-assigned, do not set this field. | string |
MedianStoppingPolicy
Name | Description | Value |
---|---|---|
policyType | [Required] Name of policy configuration | 'MedianStopping' (required) |
Microsoft.MachineLearningServices/workspaces/jobs
Name | Description | Value |
---|---|---|
name | The resource name | string Constraints: Pattern = ^[a-zA-Z0-9][a-zA-Z0-9\-_]{0,254}$ (required) |
parent | In Bicep, you can specify the parent resource for a child resource. You only need to add this property when the child resource is declared outside of the parent resource. For more information, see Child resource outside parent resource. |
Symbolic name for resource of type: workspaces |
properties | [Required] Additional attributes of the entity. | JobBase (required) |
Mpi
Name | Description | Value |
---|---|---|
distributionType | [Required] Specifies the type of distribution framework. | 'Mpi' (required) |
processCountPerInstance | Number of processes per MPI node. | int |
Objective
Name | Description | Value |
---|---|---|
goal | [Required] Defines supported metric goals for hyperparameter tuning | 'Maximize' 'Minimize' (required) |
primaryMetric | [Required] Name of the metric to optimize. | string Constraints: Pattern = [a-zA-Z0-9_] (required) |
OutputDataBinding
Name | Description | Value |
---|---|---|
datastoreId | ARM resource ID of the datastore where the data output will be stored. | string |
mode | Mechanism for data movement to datastore. | 'Direct' 'Download' 'EvalDownload' 'EvalMount' 'Mount' 'ReadOnlyMount' 'ReadWriteMount' 'Upload' |
pathOnCompute | Location of data inside the container process. | string |
pathOnDatastore | Path within the datastore to the data. | string |
PyTorch
Name | Description | Value |
---|---|---|
distributionType | [Required] Specifies the type of distribution framework. | 'PyTorch' (required) |
processCount | Total process count for the distributed job. | int |
SweepJob
Name | Description | Value |
---|---|---|
algorithm | [Required] Type of the hyperparameter sampling algorithms | 'Bayesian' 'Grid' 'Random' (required) |
compute | [Required] Compute binding for the job. | ComputeConfiguration (required) |
earlyTermination | Early termination policies enable canceling poor-performing runs before they complete. | EarlyTerminationPolicy |
experimentName | The name of the experiment the job belongs to. If not set, the job is placed in the "Default" experiment. | string |
identity | Identity configuration. If set, this should be one of AmlToken, ManagedIdentity or null. Defaults to AmlToken if null. |
IdentityConfiguration |
jobType | [Required] Specifies the type of job. | 'Sweep' (required) |
maxConcurrentTrials | An upper bound on the number of trials performed in parallel. | int |
maxTotalTrials | An upper bound on the number of trials to perform. | int |
objective | [Required] Optimization objective. | Objective (required) |
priority | Job priority for scheduling policy. Only applies to AMLCompute. Private preview feature and only available to users on the allow list. |
int |
searchSpace | [Required] A dictionary containing each parameter and its distribution. The dictionary key is the name of the parameter | SweepJobSearchSpace (required) |
timeout | The total timeout in ISO 8601 format. Only supports duration with precision as low as Minutes. | string |
trial | Trial component definition. | TrialComponent |
SweepJobSearchSpace
Name | Description | Value |
---|
TensorFlow
Name | Description | Value |
---|---|---|
distributionType | [Required] Specifies the type of distribution framework. | 'TensorFlow' (required) |
parameterServerCount | Number of parameter server tasks. | int |
workerCount | Number of workers. Overwrites the node count in compute binding. | int |
TrialComponent
Name | Description | Value |
---|---|---|
codeId | ARM resource ID of the code asset. | string |
command | [Required] The command to execute on startup of the job. eg. "python train.py" | string Constraints: Min length = 1 Pattern = [a-zA-Z0-9_] (required) |
distribution | Distribution configuration of the job. If set, this should be one of Mpi, Tensorflow, PyTorch, or null. | DistributionConfiguration |
environmentId | The ARM resource ID of the Environment specification for the job. | string |
environmentVariables | Environment variables included in the job. | TrialComponentEnvironmentVariables |
inputDataBindings | Mapping of input data bindings used in the job. | TrialComponentInputDataBindings |
outputDataBindings | Mapping of output data bindings used in the job. | TrialComponentOutputDataBindings |
timeout | The max run duration in ISO 8601 format, after which the trial component will be cancelled. Only supports duration with precision as low as Seconds. |
string |
TrialComponentEnvironmentVariables
Name | Description | Value |
---|
TrialComponentInputDataBindings
Name | Description | Value |
---|
TrialComponentOutputDataBindings
Name | Description | Value |
---|
TruncationSelectionPolicy
Name | Description | Value |
---|---|---|
policyType | [Required] Name of policy configuration | 'TruncationSelection' (required) |
truncationPercentage | The percentage of runs to cancel at each evaluation interval. | int |
Quickstart samples
The following quickstart samples deploy this resource type.
Bicep File | Description |
---|---|
Create an Azure Machine Learning AutoML classification job | This template creates an Azure Machine Learning AutoML classification job to find out the best model for predicting if a client will subscribe to a fixed term deposit with a financial institution. |
Create an Azure Machine Learning Command job | This template creates an Azure Machine Learning Command job with a basic hello_world script |
Create an Azure Machine Learning Sweep job | This template creates an Azure Machine Learning Sweep job for hyperparameter tuning. |
ARM template resource definition
The workspaces/jobs resource type can be deployed with operations that target:
- Resource groups - See resource group deployment commands
For a list of changed properties in each API version, see change log.
Resource format
To create a Microsoft.MachineLearningServices/workspaces/jobs resource, add the following JSON to your template.
{
"type": "Microsoft.MachineLearningServices/workspaces/jobs",
"apiVersion": "2021-03-01-preview",
"name": "string",
"properties": {
"description": "string",
"properties": {
"{customized property}": "string"
},
"tags": {
"{customized property}": "string"
},
"jobType": "string"
// For remaining properties, see JobBase objects
}
}
EarlyTerminationPolicy objects
Set the policyType property to specify the type of object.
For Bandit, use:
{
"policyType": "Bandit",
"slackAmount": "int",
"slackFactor": "int"
}
For MedianStopping, use:
{
"policyType": "MedianStopping"
}
For TruncationSelection, use:
{
"policyType": "TruncationSelection",
"truncationPercentage": "int"
}
IdentityConfiguration objects
Set the identityType property to specify the type of object.
For AMLToken, use:
{
"identityType": "AMLToken"
}
For Managed, use:
{
"clientId": "string",
"identityType": "Managed",
"objectId": "string",
"resourceId": "string"
}
JobBase objects
Set the jobType property to specify the type of object.
For Command, use:
{
"codeId": "string",
"command": "string",
"compute": {
"instanceCount": "int",
"instanceType": "string",
"isLocal": "bool",
"location": "string",
"properties": {
"{customized property}": "string"
},
"target": "string"
},
"distribution": {
"distributionType": "string"
// For remaining properties, see DistributionConfiguration objects
},
"environmentId": "string",
"environmentVariables": {
"{customized property}": "string"
},
"experimentName": "string",
"identity": {
"identityType": "string"
// For remaining properties, see IdentityConfiguration objects
},
"inputDataBindings": {
"{customized property}": {
"dataId": "string",
"mode": "string",
"pathOnCompute": "string"
}
},
"jobType": "Command",
"outputDataBindings": {
"{customized property}": {
"datastoreId": "string",
"mode": "string",
"pathOnCompute": "string",
"pathOnDatastore": "string"
}
},
"priority": "int",
"timeout": "string"
}
For Sweep, use:
{
"algorithm": "string",
"compute": {
"instanceCount": "int",
"instanceType": "string",
"isLocal": "bool",
"location": "string",
"properties": {
"{customized property}": "string"
},
"target": "string"
},
"earlyTermination": {
"delayEvaluation": "int",
"evaluationInterval": "int",
"policyType": "string"
// For remaining properties, see EarlyTerminationPolicy objects
},
"experimentName": "string",
"identity": {
"identityType": "string"
// For remaining properties, see IdentityConfiguration objects
},
"jobType": "Sweep",
"maxConcurrentTrials": "int",
"maxTotalTrials": "int",
"objective": {
"goal": "string",
"primaryMetric": "string"
},
"priority": "int",
"searchSpace": {
"{customized property}": {}
},
"timeout": "string",
"trial": {
"codeId": "string",
"command": "string",
"distribution": {
"distributionType": "string"
// For remaining properties, see DistributionConfiguration objects
},
"environmentId": "string",
"environmentVariables": {
"{customized property}": "string"
},
"inputDataBindings": {
"{customized property}": {
"dataId": "string",
"mode": "string",
"pathOnCompute": "string"
}
},
"outputDataBindings": {
"{customized property}": {
"datastoreId": "string",
"mode": "string",
"pathOnCompute": "string",
"pathOnDatastore": "string"
}
},
"timeout": "string"
}
}
DistributionConfiguration objects
Set the distributionType property to specify the type of object.
For Mpi, use:
{
"distributionType": "Mpi",
"processCountPerInstance": "int"
}
For PyTorch, use:
{
"distributionType": "PyTorch",
"processCount": "int"
}
For TensorFlow, use:
{
"distributionType": "TensorFlow",
"parameterServerCount": "int",
"workerCount": "int"
}
Property values
AmlToken
Name | Description | Value |
---|---|---|
identityType | [Required] Specifies the type of identity framework. | 'AMLToken' (required) |
BanditPolicy
Name | Description | Value |
---|---|---|
policyType | [Required] Name of policy configuration | 'Bandit' (required) |
slackAmount | Absolute distance allowed from the best performing run. | int |
slackFactor | Ratio of the allowed distance from the best performing run. | int |
CommandJob
Name | Description | Value |
---|---|---|
codeId | ARM resource ID of the code asset. | string |
command | [Required] The command to execute on startup of the job. eg. "python train.py" | string Constraints: Min length = 1 Pattern = [a-zA-Z0-9_] (required) |
compute | [Required] Compute binding for the job. | ComputeConfiguration (required) |
distribution | Distribution configuration of the job. If set, this should be one of Mpi, Tensorflow, PyTorch, or null. | DistributionConfiguration |
environmentId | The ARM resource ID of the Environment specification for the job. | string |
environmentVariables | Environment variables included in the job. | CommandJobEnvironmentVariables |
experimentName | The name of the experiment the job belongs to. If not set, the job is placed in the "Default" experiment. | string |
identity | Identity configuration. If set, this should be one of AmlToken, ManagedIdentity, or null. Defaults to AmlToken if null. |
IdentityConfiguration |
inputDataBindings | Mapping of input data bindings used in the job. | CommandJobInputDataBindings |
jobType | [Required] Specifies the type of job. | 'Command' (required) |
outputDataBindings | Mapping of output data bindings used in the job. | CommandJobOutputDataBindings |
priority | Job priority for scheduling policy. Only applies to AMLCompute. Private preview feature and only available to users on the allow list. |
int |
timeout | The max run duration in ISO 8601 format, after which the job will be cancelled. Only supports duration with precision as low as Seconds. | string |
CommandJobEnvironmentVariables
Name | Description | Value |
---|
CommandJobInputDataBindings
Name | Description | Value |
---|
CommandJobOutputDataBindings
Name | Description | Value |
---|
ComputeConfiguration
Name | Description | Value |
---|---|---|
instanceCount | Number of instances or nodes. | int |
instanceType | SKU type to run on. | string |
isLocal | Set to true for jobs running on local compute. | bool |
location | Location for virtual cluster run. | string |
properties | Additional properties. | ComputeConfigurationProperties |
target | ARM resource ID of the Compute you are targeting. If not provided the resource will be deployed as Managed. | string |
ComputeConfigurationProperties
Name | Description | Value |
---|
DistributionConfiguration
Name | Description | Value |
---|---|---|
distributionType | Set to 'Mpi' for type Mpi. Set to 'PyTorch' for type PyTorch. Set to 'TensorFlow' for type TensorFlow. | 'Mpi' 'PyTorch' 'TensorFlow' (required) |
EarlyTerminationPolicy
Name | Description | Value |
---|---|---|
delayEvaluation | Number of intervals by which to delay the first evaluation. | int |
evaluationInterval | Interval (number of runs) between policy evaluations. | int |
policyType | Set to 'Bandit' for type BanditPolicy. Set to 'MedianStopping' for type MedianStoppingPolicy. Set to 'TruncationSelection' for type TruncationSelectionPolicy. | 'Bandit' 'MedianStopping' 'TruncationSelection' (required) |
IdentityConfiguration
Name | Description | Value |
---|---|---|
identityType | Set to 'AMLToken' for type AmlToken. Set to 'Managed' for type ManagedIdentity. | 'AMLToken' 'Managed' (required) |
InputDataBinding
Name | Description | Value |
---|---|---|
dataId | ARM resource ID of the registered dataVersion. | string |
mode | Mechanism for accessing the data artifact. | 'Direct' 'Download' 'EvalDownload' 'EvalMount' 'Mount' 'ReadOnlyMount' 'ReadWriteMount' 'Upload' |
pathOnCompute | Location of data inside the container process. | string |
JobBase
Name | Description | Value |
---|---|---|
description | The asset description text. | string |
jobType | Set to 'Command' for type CommandJob. Set to 'Sweep' for type SweepJob. | 'Command' 'Sweep' (required) |
properties | The asset property dictionary. | JobBaseProperties |
tags | Tag dictionary. Tags can be added, removed, and updated. | JobBaseTags |
JobBaseProperties
Name | Description | Value |
---|
JobBaseTags
Name | Description | Value |
---|
ManagedIdentity
Name | Description | Value |
---|---|---|
clientId | Specifies a user-assigned identity by client ID. For system-assigned, do not set this field. | string Constraints: Min length = 36 Max length = 36 Pattern = ^[0-9a-fA-F]{8}-([0-9a-fA-F]{4}-){3}[0-9a-fA-F]{12}$ |
identityType | [Required] Specifies the type of identity framework. | 'Managed' (required) |
objectId | Specifies a user-assigned identity by object ID. For system-assigned, do not set this field. | string Constraints: Min length = 36 Max length = 36 Pattern = ^[0-9a-fA-F]{8}-([0-9a-fA-F]{4}-){3}[0-9a-fA-F]{12}$ |
resourceId | Specifies a user-assigned identity by ARM resource ID. For system-assigned, do not set this field. | string |
MedianStoppingPolicy
Name | Description | Value |
---|---|---|
policyType | [Required] Name of policy configuration | 'MedianStopping' (required) |
Microsoft.MachineLearningServices/workspaces/jobs
Name | Description | Value |
---|---|---|
apiVersion | The api version | '2021-03-01-preview' |
name | The resource name | string Constraints: Pattern = ^[a-zA-Z0-9][a-zA-Z0-9\-_]{0,254}$ (required) |
properties | [Required] Additional attributes of the entity. | JobBase (required) |
type | The resource type | 'Microsoft.MachineLearningServices/workspaces/jobs' |
Mpi
Name | Description | Value |
---|---|---|
distributionType | [Required] Specifies the type of distribution framework. | 'Mpi' (required) |
processCountPerInstance | Number of processes per MPI node. | int |
Objective
Name | Description | Value |
---|---|---|
goal | [Required] Defines supported metric goals for hyperparameter tuning | 'Maximize' 'Minimize' (required) |
primaryMetric | [Required] Name of the metric to optimize. | string Constraints: Pattern = [a-zA-Z0-9_] (required) |
OutputDataBinding
Name | Description | Value |
---|---|---|
datastoreId | ARM resource ID of the datastore where the data output will be stored. | string |
mode | Mechanism for data movement to datastore. | 'Direct' 'Download' 'EvalDownload' 'EvalMount' 'Mount' 'ReadOnlyMount' 'ReadWriteMount' 'Upload' |
pathOnCompute | Location of data inside the container process. | string |
pathOnDatastore | Path within the datastore to the data. | string |
PyTorch
Name | Description | Value |
---|---|---|
distributionType | [Required] Specifies the type of distribution framework. | 'PyTorch' (required) |
processCount | Total process count for the distributed job. | int |
SweepJob
Name | Description | Value |
---|---|---|
algorithm | [Required] Type of the hyperparameter sampling algorithms | 'Bayesian' 'Grid' 'Random' (required) |
compute | [Required] Compute binding for the job. | ComputeConfiguration (required) |
earlyTermination | Early termination policies enable canceling poor-performing runs before they complete. | EarlyTerminationPolicy |
experimentName | The name of the experiment the job belongs to. If not set, the job is placed in the "Default" experiment. | string |
identity | Identity configuration. If set, this should be one of AmlToken, ManagedIdentity or null. Defaults to AmlToken if null. |
IdentityConfiguration |
jobType | [Required] Specifies the type of job. | 'Sweep' (required) |
maxConcurrentTrials | An upper bound on the number of trials performed in parallel. | int |
maxTotalTrials | An upper bound on the number of trials to perform. | int |
objective | [Required] Optimization objective. | Objective (required) |
priority | Job priority for scheduling policy. Only applies to AMLCompute. Private preview feature and only available to users on the allow list. |
int |
searchSpace | [Required] A dictionary containing each parameter and its distribution. The dictionary key is the name of the parameter | SweepJobSearchSpace (required) |
timeout | The total timeout in ISO 8601 format. Only supports duration with precision as low as Minutes. | string |
trial | Trial component definition. | TrialComponent |
SweepJobSearchSpace
Name | Description | Value |
---|
TensorFlow
Name | Description | Value |
---|---|---|
distributionType | [Required] Specifies the type of distribution framework. | 'TensorFlow' (required) |
parameterServerCount | Number of parameter server tasks. | int |
workerCount | Number of workers. Overwrites the node count in compute binding. | int |
TrialComponent
Name | Description | Value |
---|---|---|
codeId | ARM resource ID of the code asset. | string |
command | [Required] The command to execute on startup of the job. eg. "python train.py" | string Constraints: Min length = 1 Pattern = [a-zA-Z0-9_] (required) |
distribution | Distribution configuration of the job. If set, this should be one of Mpi, Tensorflow, PyTorch, or null. | DistributionConfiguration |
environmentId | The ARM resource ID of the Environment specification for the job. | string |
environmentVariables | Environment variables included in the job. | TrialComponentEnvironmentVariables |
inputDataBindings | Mapping of input data bindings used in the job. | TrialComponentInputDataBindings |
outputDataBindings | Mapping of output data bindings used in the job. | TrialComponentOutputDataBindings |
timeout | The max run duration in ISO 8601 format, after which the trial component will be cancelled. Only supports duration with precision as low as Seconds. |
string |
TrialComponentEnvironmentVariables
Name | Description | Value |
---|
TrialComponentInputDataBindings
Name | Description | Value |
---|
TrialComponentOutputDataBindings
Name | Description | Value |
---|
TruncationSelectionPolicy
Name | Description | Value |
---|---|---|
policyType | [Required] Name of policy configuration | 'TruncationSelection' (required) |
truncationPercentage | The percentage of runs to cancel at each evaluation interval. | int |
Quickstart templates
The following quickstart templates deploy this resource type.
Template | Description |
---|---|
Create an Azure Machine Learning AutoML classification job |
This template creates an Azure Machine Learning AutoML classification job to find out the best model for predicting if a client will subscribe to a fixed term deposit with a financial institution. |
Create an Azure Machine Learning Command job |
This template creates an Azure Machine Learning Command job with a basic hello_world script |
Create an Azure Machine Learning Sweep job |
This template creates an Azure Machine Learning Sweep job for hyperparameter tuning. |
Terraform (AzAPI provider) resource definition
The workspaces/jobs resource type can be deployed with operations that target:
- Resource groups
For a list of changed properties in each API version, see change log.
Resource format
To create a Microsoft.MachineLearningServices/workspaces/jobs resource, add the following Terraform to your template.
resource "azapi_resource" "symbolicname" {
type = "Microsoft.MachineLearningServices/workspaces/jobs@2021-03-01-preview"
name = "string"
body = jsonencode({
properties = {
description = "string"
properties = {
{customized property} = "string"
}
tags = {
{customized property} = "string"
}
jobType = "string"
// For remaining properties, see JobBase objects
}
})
}
EarlyTerminationPolicy objects
Set the policyType property to specify the type of object.
For Bandit, use:
{
policyType = "Bandit"
slackAmount = int
slackFactor = int
}
For MedianStopping, use:
{
policyType = "MedianStopping"
}
For TruncationSelection, use:
{
policyType = "TruncationSelection"
truncationPercentage = int
}
IdentityConfiguration objects
Set the identityType property to specify the type of object.
For AMLToken, use:
{
identityType = "AMLToken"
}
For Managed, use:
{
clientId = "string"
identityType = "Managed"
objectId = "string"
resourceId = "string"
}
JobBase objects
Set the jobType property to specify the type of object.
For Command, use:
{
codeId = "string"
command = "string"
compute = {
instanceCount = int
instanceType = "string"
isLocal = bool
location = "string"
properties = {
{customized property} = "string"
}
target = "string"
}
distribution = {
distributionType = "string"
// For remaining properties, see DistributionConfiguration objects
}
environmentId = "string"
environmentVariables = {
{customized property} = "string"
}
experimentName = "string"
identity = {
identityType = "string"
// For remaining properties, see IdentityConfiguration objects
}
inputDataBindings = {
{customized property} = {
dataId = "string"
mode = "string"
pathOnCompute = "string"
}
}
jobType = "Command"
outputDataBindings = {
{customized property} = {
datastoreId = "string"
mode = "string"
pathOnCompute = "string"
pathOnDatastore = "string"
}
}
priority = int
timeout = "string"
}
For Sweep, use:
{
algorithm = "string"
compute = {
instanceCount = int
instanceType = "string"
isLocal = bool
location = "string"
properties = {
{customized property} = "string"
}
target = "string"
}
earlyTermination = {
delayEvaluation = int
evaluationInterval = int
policyType = "string"
// For remaining properties, see EarlyTerminationPolicy objects
}
experimentName = "string"
identity = {
identityType = "string"
// For remaining properties, see IdentityConfiguration objects
}
jobType = "Sweep"
maxConcurrentTrials = int
maxTotalTrials = int
objective = {
goal = "string"
primaryMetric = "string"
}
priority = int
searchSpace = {
{customized property} = ?
}
timeout = "string"
trial = {
codeId = "string"
command = "string"
distribution = {
distributionType = "string"
// For remaining properties, see DistributionConfiguration objects
}
environmentId = "string"
environmentVariables = {
{customized property} = "string"
}
inputDataBindings = {
{customized property} = {
dataId = "string"
mode = "string"
pathOnCompute = "string"
}
}
outputDataBindings = {
{customized property} = {
datastoreId = "string"
mode = "string"
pathOnCompute = "string"
pathOnDatastore = "string"
}
}
timeout = "string"
}
}
DistributionConfiguration objects
Set the distributionType property to specify the type of object.
For Mpi, use:
{
distributionType = "Mpi"
processCountPerInstance = int
}
For PyTorch, use:
{
distributionType = "PyTorch"
processCount = int
}
For TensorFlow, use:
{
distributionType = "TensorFlow"
parameterServerCount = int
workerCount = int
}
Property values
AmlToken
Name | Description | Value |
---|---|---|
identityType | [Required] Specifies the type of identity framework. | 'AMLToken' (required) |
BanditPolicy
Name | Description | Value |
---|---|---|
policyType | [Required] Name of policy configuration | 'Bandit' (required) |
slackAmount | Absolute distance allowed from the best performing run. | int |
slackFactor | Ratio of the allowed distance from the best performing run. | int |
CommandJob
Name | Description | Value |
---|---|---|
codeId | ARM resource ID of the code asset. | string |
command | [Required] The command to execute on startup of the job. eg. "python train.py" | string Constraints: Min length = 1 Pattern = [a-zA-Z0-9_] (required) |
compute | [Required] Compute binding for the job. | ComputeConfiguration (required) |
distribution | Distribution configuration of the job. If set, this should be one of Mpi, Tensorflow, PyTorch, or null. | DistributionConfiguration |
environmentId | The ARM resource ID of the Environment specification for the job. | string |
environmentVariables | Environment variables included in the job. | CommandJobEnvironmentVariables |
experimentName | The name of the experiment the job belongs to. If not set, the job is placed in the "Default" experiment. | string |
identity | Identity configuration. If set, this should be one of AmlToken, ManagedIdentity, or null. Defaults to AmlToken if null. |
IdentityConfiguration |
inputDataBindings | Mapping of input data bindings used in the job. | CommandJobInputDataBindings |
jobType | [Required] Specifies the type of job. | 'Command' (required) |
outputDataBindings | Mapping of output data bindings used in the job. | CommandJobOutputDataBindings |
priority | Job priority for scheduling policy. Only applies to AMLCompute. Private preview feature and only available to users on the allow list. |
int |
timeout | The max run duration in ISO 8601 format, after which the job will be cancelled. Only supports duration with precision as low as Seconds. | string |
CommandJobEnvironmentVariables
Name | Description | Value |
---|
CommandJobInputDataBindings
Name | Description | Value |
---|
CommandJobOutputDataBindings
Name | Description | Value |
---|
ComputeConfiguration
Name | Description | Value |
---|---|---|
instanceCount | Number of instances or nodes. | int |
instanceType | SKU type to run on. | string |
isLocal | Set to true for jobs running on local compute. | bool |
location | Location for virtual cluster run. | string |
properties | Additional properties. | ComputeConfigurationProperties |
target | ARM resource ID of the Compute you are targeting. If not provided the resource will be deployed as Managed. | string |
ComputeConfigurationProperties
Name | Description | Value |
---|
DistributionConfiguration
Name | Description | Value |
---|---|---|
distributionType | Set to 'Mpi' for type Mpi. Set to 'PyTorch' for type PyTorch. Set to 'TensorFlow' for type TensorFlow. | 'Mpi' 'PyTorch' 'TensorFlow' (required) |
EarlyTerminationPolicy
Name | Description | Value |
---|---|---|
delayEvaluation | Number of intervals by which to delay the first evaluation. | int |
evaluationInterval | Interval (number of runs) between policy evaluations. | int |
policyType | Set to 'Bandit' for type BanditPolicy. Set to 'MedianStopping' for type MedianStoppingPolicy. Set to 'TruncationSelection' for type TruncationSelectionPolicy. | 'Bandit' 'MedianStopping' 'TruncationSelection' (required) |
IdentityConfiguration
Name | Description | Value |
---|---|---|
identityType | Set to 'AMLToken' for type AmlToken. Set to 'Managed' for type ManagedIdentity. | 'AMLToken' 'Managed' (required) |
InputDataBinding
Name | Description | Value |
---|---|---|
dataId | ARM resource ID of the registered dataVersion. | string |
mode | Mechanism for accessing the data artifact. | 'Direct' 'Download' 'EvalDownload' 'EvalMount' 'Mount' 'ReadOnlyMount' 'ReadWriteMount' 'Upload' |
pathOnCompute | Location of data inside the container process. | string |
JobBase
Name | Description | Value |
---|---|---|
description | The asset description text. | string |
jobType | Set to 'Command' for type CommandJob. Set to 'Sweep' for type SweepJob. | 'Command' 'Sweep' (required) |
properties | The asset property dictionary. | JobBaseProperties |
tags | Tag dictionary. Tags can be added, removed, and updated. | JobBaseTags |
JobBaseProperties
Name | Description | Value |
---|
JobBaseTags
Name | Description | Value |
---|
ManagedIdentity
Name | Description | Value |
---|---|---|
clientId | Specifies a user-assigned identity by client ID. For system-assigned, do not set this field. | string Constraints: Min length = 36 Max length = 36 Pattern = ^[0-9a-fA-F]{8}-([0-9a-fA-F]{4}-){3}[0-9a-fA-F]{12}$ |
identityType | [Required] Specifies the type of identity framework. | 'Managed' (required) |
objectId | Specifies a user-assigned identity by object ID. For system-assigned, do not set this field. | string Constraints: Min length = 36 Max length = 36 Pattern = ^[0-9a-fA-F]{8}-([0-9a-fA-F]{4}-){3}[0-9a-fA-F]{12}$ |
resourceId | Specifies a user-assigned identity by ARM resource ID. For system-assigned, do not set this field. | string |
MedianStoppingPolicy
Name | Description | Value |
---|---|---|
policyType | [Required] Name of policy configuration | 'MedianStopping' (required) |
Microsoft.MachineLearningServices/workspaces/jobs
Name | Description | Value |
---|---|---|
name | The resource name | string Constraints: Pattern = ^[a-zA-Z0-9][a-zA-Z0-9\-_]{0,254}$ (required) |
parent_id | The ID of the resource that is the parent for this resource. | ID for resource of type: workspaces |
properties | [Required] Additional attributes of the entity. | JobBase (required) |
type | The resource type | "Microsoft.MachineLearningServices/workspaces/jobs@2021-03-01-preview" |
Mpi
Name | Description | Value |
---|---|---|
distributionType | [Required] Specifies the type of distribution framework. | 'Mpi' (required) |
processCountPerInstance | Number of processes per MPI node. | int |
Objective
Name | Description | Value |
---|---|---|
goal | [Required] Defines supported metric goals for hyperparameter tuning | 'Maximize' 'Minimize' (required) |
primaryMetric | [Required] Name of the metric to optimize. | string Constraints: Pattern = [a-zA-Z0-9_] (required) |
OutputDataBinding
Name | Description | Value |
---|---|---|
datastoreId | ARM resource ID of the datastore where the data output will be stored. | string |
mode | Mechanism for data movement to datastore. | 'Direct' 'Download' 'EvalDownload' 'EvalMount' 'Mount' 'ReadOnlyMount' 'ReadWriteMount' 'Upload' |
pathOnCompute | Location of data inside the container process. | string |
pathOnDatastore | Path within the datastore to the data. | string |
PyTorch
Name | Description | Value |
---|---|---|
distributionType | [Required] Specifies the type of distribution framework. | 'PyTorch' (required) |
processCount | Total process count for the distributed job. | int |
SweepJob
Name | Description | Value |
---|---|---|
algorithm | [Required] Type of the hyperparameter sampling algorithms | 'Bayesian' 'Grid' 'Random' (required) |
compute | [Required] Compute binding for the job. | ComputeConfiguration (required) |
earlyTermination | Early termination policies enable canceling poor-performing runs before they complete. | EarlyTerminationPolicy |
experimentName | The name of the experiment the job belongs to. If not set, the job is placed in the "Default" experiment. | string |
identity | Identity configuration. If set, this should be one of AmlToken, ManagedIdentity or null. Defaults to AmlToken if null. |
IdentityConfiguration |
jobType | [Required] Specifies the type of job. | 'Sweep' (required) |
maxConcurrentTrials | An upper bound on the number of trials performed in parallel. | int |
maxTotalTrials | An upper bound on the number of trials to perform. | int |
objective | [Required] Optimization objective. | Objective (required) |
priority | Job priority for scheduling policy. Only applies to AMLCompute. Private preview feature and only available to users on the allow list. |
int |
searchSpace | [Required] A dictionary containing each parameter and its distribution. The dictionary key is the name of the parameter | SweepJobSearchSpace (required) |
timeout | The total timeout in ISO 8601 format. Only supports duration with precision as low as Minutes. | string |
trial | Trial component definition. | TrialComponent |
SweepJobSearchSpace
Name | Description | Value |
---|
TensorFlow
Name | Description | Value |
---|---|---|
distributionType | [Required] Specifies the type of distribution framework. | 'TensorFlow' (required) |
parameterServerCount | Number of parameter server tasks. | int |
workerCount | Number of workers. Overwrites the node count in compute binding. | int |
TrialComponent
Name | Description | Value |
---|---|---|
codeId | ARM resource ID of the code asset. | string |
command | [Required] The command to execute on startup of the job. eg. "python train.py" | string Constraints: Min length = 1 Pattern = [a-zA-Z0-9_] (required) |
distribution | Distribution configuration of the job. If set, this should be one of Mpi, Tensorflow, PyTorch, or null. | DistributionConfiguration |
environmentId | The ARM resource ID of the Environment specification for the job. | string |
environmentVariables | Environment variables included in the job. | TrialComponentEnvironmentVariables |
inputDataBindings | Mapping of input data bindings used in the job. | TrialComponentInputDataBindings |
outputDataBindings | Mapping of output data bindings used in the job. | TrialComponentOutputDataBindings |
timeout | The max run duration in ISO 8601 format, after which the trial component will be cancelled. Only supports duration with precision as low as Seconds. |
string |
TrialComponentEnvironmentVariables
Name | Description | Value |
---|
TrialComponentInputDataBindings
Name | Description | Value |
---|
TrialComponentOutputDataBindings
Name | Description | Value |
---|
TruncationSelectionPolicy
Name | Description | Value |
---|---|---|
policyType | [Required] Name of policy configuration | 'TruncationSelection' (required) |
truncationPercentage | The percentage of runs to cancel at each evaluation interval. | int |