model_customization Package

Classes

EndpointRequestSettings	Note This is an experimental class, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information. Initialize EndpointRequestSettings.
PromptSettings	Note This is an experimental class, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information. Initialize PromptSettings.
TeacherModelSettings	Note This is an experimental class, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information. Initialize TeacherModelSettings

Functions

distillation

Note

This is an experimental method, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information.

Function to create a Distillation job.

A distillation job is used to transfer knowledge from a teacher model to student model by a two step process of generating synthetic data from the teacher model and then finetuning the student model with the generated synthetic data.

distillation(*, experiment_name: str, data_generation_type: str, data_generation_task_type: str, teacher_model_endpoint_connection: WorkspaceConnection, student_model: Input | str, training_data: Input | str | None = None, validation_data: Input | str | None = None, teacher_model_settings: TeacherModelSettings | None = None, prompt_settings: PromptSettings | None = None, hyperparameters: Dict | None = None, resources: ResourceConfiguration | None = None, **kwargs: Any) -> DistillationJob

Parameters

Name	Description
experiment_name Required	str The name of the experiment.
data_generation_type Required	str The type of data generation to perform. Acceptable values: label_generation
data_generation_task_type Required	The type of data to generate Acceptable values: NLI, NLU_QA, CONVERSATION, MATH, SUMMARIZATION
teacher_model_endpoint_connection Required	The kind of teacher model connection that includes the name, endpoint url, and api_key.
student_model Required	Union[Input, str] The model to train
training_data Required	Optional[Union[Input, str]], <xref:optional> The training data to use. Should contain the questions but not the labels, defaults to None
validation_data Required	Optional[Union[Input, str]], <xref:optional> The validation data to use. Should contain the questions but not the labels, defaults to None
teacher_model_settings Required	Optional[TeacherModelSettings], <xref:optional> The settings for the teacher model. Accepts both the inference parameters and endpoint settings, defaults to None Acceptable keys for inference parameters: temperature, max_tokens, top_p, frequency_penalty, presence_penalty, stop
prompt_settings Required	Optional[PromptSettings], <xref:optional> The settings for the prompt that affect the system prompt used for data generation, defaults to None
hyperparameters Required	Optional[Dict], <xref:optional> The hyperparameters to use for finetuning, defaults to None
resources Required	Optional[ResourceConfiguration], <xref:optional> The compute resource to use for the data generation step in the distillation job, defaults to None

Returns

Type	Description
<xref:DistillationJob>	A DistillationJob to submit

Exceptions

Type	Description
ValueError	Raises ValueError if there is no training data and data generation type is 'label_generation'