InferenceConfig Class

Reference

Represents configuration settings for a custom environment used for deployment.

Inference configuration is an input parameter for Model deployment-related actions:

Initialize the config object.

Inheritance: builtins.object

InferenceConfig

Constructor

InferenceConfig(entry_script, runtime=None, conda_file=None, extra_docker_file_steps=None, source_directory=None, enable_gpu=None, description=None, base_image=None, base_image_registry=None, cuda_version=None, environment=None)

Parameters

Name	Description
entry_script Required	str The path to a local file that contains the code to run for the image.
runtime	str The runtime to use for the image. Current supported runtimes are 'spark-py' and 'python'. Default value: None
conda_file	str The path to a local file containing a conda environment definition to use for the image. Default value: None
extra_docker_file_steps	str The path to a local file containing additional Docker steps to run when setting up image. Default value: None
source_directory	str The path to the folder that contains all files to create the image. Default value: None
enable_gpu	bool Indicates whether to enable GPU support in the image. The GPU image must be used on Microsoft Azure Services such as Azure Container Instances, Azure Machine Learning Compute, Azure Virtual Machines, and Azure Kubernetes Service. Defaults to False. Default value: None
description	str A description to give this image. Default value: None
base_image	str A custom image to be used as base image. If no base image is given then the base image will be used based off of given runtime parameter. Default value: None
base_image_registry	ContainerRegistry The image registry that contains the base image. Default value: None
cuda_version	str The Version of CUDA to install for images that need GPU support. The GPU image must be used on Microsoft Azure Services such as Azure Container Instances, Azure Machine Learning Compute, Azure Virtual Machines, and Azure Kubernetes Service. Supported versions are 9.0, 9.1, and 10.0. If `enable_gpu` is set, this defaults to '9.1'. Default value: None
environment	Environment An environment object to use for the deployment. The environment doesn't have to be registered. Provide either this parameter, or the other parameters, but not both. The individual parameters will NOT serve as an override for the environment object. Exceptions include `entry_script`, `source_directory`, and `description`. Default value: None
entry_script Required	str The path to a local file that contains the code to run for the image.
runtime Required	str The runtime to use for the image. Current supported runtimes are 'spark-py' and 'python'.
conda_file Required	str The path to a local file containing a conda environment definition to use for the image.
extra_docker_file_steps Required	str The path to a local file containing additional Docker steps to run when setting up image.
source_directory Required	str The path to the folder that contains all files to create the image.
enable_gpu Required	bool Indicates whether to enable GPU support in the image. The GPU image must be used on Microsoft Azure Services such as Azure Container Instances, Azure Machine Learning Compute, Azure Virtual Machines, and Azure Kubernetes Service. Defaults to False.
description Required	str A description to give this image.
base_image Required	str A custom image to be used as base image. If no base image is given then the base image will be used based off of given runtime parameter.
base_image_registry Required	ContainerRegistry The image registry that contains the base image.
cuda_version Required	str The Version of CUDA to install for images that need GPU support. The GPU image must be used on Microsoft Azure Services such as Azure Container Instances, Azure Machine Learning Compute, Azure Virtual Machines, and Azure Kubernetes Service. Supported versions are 9.0, 9.1, and 10.0. If `enable_gpu` is set, this defaults to '9.1'.
environment Required	Environment An environment object to use for the deployment. The environment doesn't have to be registered. Provide either this parameter, or the other parameters, but not both. The individual parameters will NOT serve as an override for the environment object. Exceptions include `entry_script`, `source_directory`, and `description`.

Remarks

The following sample shows how to create an InferenceConfig object and use it to deploy a model.


   from azureml.core.model import InferenceConfig
   from azureml.core.webservice import AciWebservice


   service_name = 'my-custom-env-service'

   inference_config = InferenceConfig(entry_script='score.py', environment=environment)
   aci_config = AciWebservice.deploy_configuration(cpu_cores=1, memory_gb=1)

   service = Model.deploy(workspace=ws,
                          name=service_name,
                          models=[model],
                          inference_config=inference_config,
                          deployment_config=aci_config,
                          overwrite=True)
   service.wait_for_deployment(show_output=True)

Variables

Name	Description
entry_script	str The path to a local file that contains the code to run for the image.
runtime	str The runtime to use for the image. Current supported runtimes are 'spark-py' and 'python'.
conda_file	str The path to a local file containing a conda environment definition to use for the image.
extra_docker_file_steps	str The path to a local file containing additional Docker steps to run when setting up the image.
source_directory	str The path to the folder that contains all files to create the image.
enable_gpu	bool Indicates whether to enable GPU support in the image. The GPU image must be used on Microsoft Azure Services such as Azure Container Instances, Azure Machine Learning Compute, Azure Virtual Machines, and Azure Kubernetes Service.
azureml.core.model.InferenceConfig.description	A description to give this image.
base_image	str A custom image to be used as base image. If no base image is given then the base image will be used based off of given runtime parameter.
base_image_registry	ContainerRegistry The image registry that contains the base image.
cuda_version	str The version of CUDA to install for images that need GPU support. The GPU image must be used on Microsoft Azure Services such as Azure Container Instances, Azure Machine Learning Compute, Azure Virtual Machines, and Azure Kubernetes Service. Supported versions are 9.0, 9.1, and 10.0. If `enable_gpu` is set, this defaults to '9.1'.
azureml.core.model.InferenceConfig.environment	An environment object to use for the deployment. The environment doesn't have to be registered. Provide either this parameter, or the other parameters, but not both. The individual parameters will NOT serve as an override for the environment object. Exceptions include `entry_script`, `source_directory`, and `description`.

Methods

build_create_payload	Build the creation payload for the Container image.
build_profile_payload	Build the profiling payload for the Model package.
validate_configuration	Check that the specified configuration values are valid. Raises a WebserviceException if validation fails.
validation_script_content	Check that the syntax of score script is valid with ast.parse. Raises a UserErrorException if validation fails.

build_create_payload

Build the creation payload for the Container image.

build_create_payload(workspace, name, model_ids)

Parameters

Name	Description
workspace Required	Workspace The workspace object to create the image in.
name Required	str The name of the image.
model_ids Required	list[str] A list of model IDs to package into the image.

Returns

Type	Description
dict	The container image creation payload.

Exceptions

Type	Description
WebserviceException

build_profile_payload

Build the profiling payload for the Model package.

build_profile_payload(profile_name, input_data=None, workspace=None, models=None, dataset_id=None, container_resource_requirements=None, description=None)

Parameters

Name	Description
profile_name Required	str The name of the profiling run.
input_data	str The input data for profiling. Default value: None
workspace	Workspace A Workspace object in which to profile the model. Default value: None
models	list[Model] A list of model objects. Can be an empty list. Default value: None
dataset_id	str Id associated with the dataset containing input data for the profiling run. Default value: None
container_resource_requirements	ContainerResourceRequirements container resource requirements for the largest instance to which the model is to be deployed Default value: None
description	str Description to be associated with the profiling run. Default value: None

Returns

Type	Description
dict	Model profile payload

Exceptions

Type	Description
WebserviceException

validate_configuration

Check that the specified configuration values are valid.

Raises a WebserviceException if validation fails.

validate_configuration()

Exceptions

Type	Description
WebserviceException

validation_script_content

Check that the syntax of score script is valid with ast.parse.

Raises a UserErrorException if validation fails.

validation_script_content()

Exceptions

Type	Description
UserErrorException

Share via

InferenceConfig Class

Constructor

Parameters

Remarks

Variables

Methods

build_create_payload

Parameters

Returns

Exceptions

build_profile_payload

Parameters

Returns

Exceptions

validate_configuration

Exceptions

validation_script_content

Exceptions

Feedback

Additional resources