Share via

output_dataset_config Module

  • Reference

Contains configurations that specifies how outputs for a job should be uploaded and promoted to a dataset.

For more information, see the article how to specify outputs.

Classes

HDFSOutputDatasetConfig

Represent how to output to a HDFS path and be promoted as a FileDataset.

Initialize a HDFSOutputDatasetConfig.
LinkFileOutputDatasetConfig

Note

This is an experimental class, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information.

Represent how to link the output of a run and be promoted as a FileDataset.

The LinkFileOutputDatasetConfig allows you to link a file dataset as output dataset


   workspace = Workspace.from_config()
   experiment = Experiment(workspace, 'output_example')

   output = LinkFileOutputDatasetConfig('link_output')

   script_run_config = ScriptRunConfig('.', 'link.py', arguments=[output])

   # within link.py
   # from azureml.core import Run, Dataset
   # run = Run.get_context()
   # workspace = run.experiment.workspace
   # dataset = Dataset.get_by_name(workspace, name='dataset_to_link')
   # run.output_datasets['link_output'].link(dataset)

   run = experiment.submit(script_run_config)
   print(run)

Initialize a LinkFileOutputDatasetConfig.
LinkTabularOutputDatasetConfig

Note

This is an experimental class, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information.

Represent how to link the output of a run and be promoted as a TabularDataset.

The LinkTabularOutputDatasetConfig allows you to link a file Tabular as output dataset


   workspace = Workspace.from_config()
   experiment = Experiment(workspace, 'output_example')

   output = LinkTabularOutputDatasetConfig('link_output')

   script_run_config = ScriptRunConfig('.', 'link.py', arguments=[output])

   # within link.py
   # from azureml.core import Run, Dataset
   # run = Run.get_context()
   # workspace = run.experiment.workspace
   # dataset = Dataset.get_by_name(workspace, name='dataset_to_link')
   # run.output_datasets['link_output'].link(dataset)

   run = experiment.submit(script_run_config)
   print(run)

Initialize a LinkTabularOutputDatasetConfig.
MountOptions

Options that are specific to output which will be mounted.

Initialize a MountOptions.
OutputDatasetConfig

Represent how to copy the output of a job to a remote storage location and be promoted to a Dataset.

This is the base class used to represent how to copy the output of a job to a remote storage location, whether to register it as a named and versioned Dataset, and whether to apply any additional transformations to the Dataset that was created.

You should not be creating instances of this class directly but instead should use the appropriate subclass.

Initialize a OutputDatasetConfig.
OutputFileDatasetConfig

Represent how to copy the output of a run and be promoted as a FileDataset.

The OutputFileDatasetConfig allows you to specify how you want a particular local path on the compute target to be uploaded to the specified destination. If no arguments are passed to the constructor, we will automatically generate a name, a destination, and a local path.

An example of not passing any arguments:


   workspace = Workspace.from_config()
   experiment = Experiment(workspace, 'output_example')

   output = OutputFileDatasetConfig()

   script_run_config = ScriptRunConfig('.', 'train.py', arguments=[output])

   run = experiment.submit(script_run_config)
   print(run)

An example of creating an output then promoting the output to a tabular dataset and register it with name foo:


   workspace = Workspace.from_config()
   experiment = Experiment(workspace, 'output_example')

   datastore = Datastore(workspace, 'example_adls_gen2_datastore')

   # for more information on the parameters and methods, please look for the corresponding documentation.
   output = OutputFileDatasetConfig().read_delimited_files().register_on_complete('foo')

   script_run_config = ScriptRunConfig('.', 'train.py', arguments=[output])

   run = experiment.submit(script_run_config)
   print(run)

Initialize a OutputFileDatasetConfig.

The OutputFileDatasetConfig allows you to specify how you want a particular local path on the compute target to be uploaded to the specified destination. If no arguments are passed to the constructor, we will automatically generate a name, a destination, and a local path.

An example of not passing any arguments:


   workspace = Workspace.from_config()
   experiment = Experiment(workspace, 'output_example')

   output = OutputFileDatasetConfig()

   script_run_config = ScriptRunConfig('.', 'train.py', arguments=[output])

   run = experiment.submit(script_run_config)
   print(run)

An example of creating an output then promoting the output to a tabular dataset and register it with name foo:


   workspace = Workspace.from_config()
   experiment = Experiment(workspace, 'output_example')

   datastore = Datastore(workspace, 'example_adls_gen2_datastore')

   # for more information on the parameters and methods, please look for the corresponding documentation.
   output = OutputFileDatasetConfig().read_delimited_files().register_on_complete('foo')

   script_run_config = ScriptRunConfig('.', 'train.py', arguments=[output])

   run = experiment.submit(script_run_config)
   print(run)
OutputTabularDatasetConfig

Represent how to copy the output of a run and be promoted as a TabularDataset.

Initialize a OutputTabularDatasetConfig.
RegistrationConfiguration

Configuration that specifies how to register the output as a Dataset.

Initialize a RegistrationConfiguration.
TransformationMixin

This class provides transformation capabilities to output datasets.
UploadOptions

Options that are specific to output which will be uploaded.

Initialize a UploadOptions.