BanditPolicy Class

Defines an early termination policy based on slack criteria and a frequency and delay interval for evaluation.

Constructor

BanditPolicy(*, delay_evaluation: int = 0, evaluation_interval: int = 0, slack_amount: float = 0, slack_factor: float = 0)

Keyword-Only Parameters

Name	Description
delay_evaluation	int Number of intervals by which to delay the first evaluation. Defaults to 0.
evaluation_interval	int Interval (number of runs) between policy evaluations. Defaults to 0.
slack_amount	float Absolute distance allowed from the best performing run. Defaults to 0.
slack_factor	float Ratio of the allowed distance from the best performing run. Defaults to 0.

Examples

Configuring BanditPolicy early termination of a hyperparameter sweep on a Command job.

   from azure.ai.ml import command

   job = command(
       inputs=dict(kernel="linear", penalty=1.0),
       compute=cpu_cluster,
       environment=f"{job_env.name}:{job_env.version}",
       code="./scripts",
       command="python scripts/train.py --kernel $kernel --penalty $penalty",
       experiment_name="sklearn-iris-flowers",
   )

   # we can reuse an existing Command Job as a function that we can apply inputs to for the sweep configurations
   from azure.ai.ml.sweep import Uniform

   job_for_sweep = job(
       kernel=Uniform(min_value=0.0005, max_value=0.005),
       penalty=Uniform(min_value=0.9, max_value=0.99),
   )

   from azure.ai.ml.sweep import BanditPolicy

   sweep_job = job_for_sweep.sweep(
       sampling_algorithm="random",
       primary_metric="best_val_acc",
       goal="Maximize",
       max_total_trials=8,
       max_concurrent_trials=4,
       early_termination_policy=BanditPolicy(slack_factor=0.15, evaluation_interval=1, delay_evaluation=10),
   )