ScenarioRun Class

Definition

Namespace:

Microsoft.Extensions.AI.Evaluation.Reporting

Assembly:

Microsoft.Extensions.AI.Evaluation.Reporting.dll

Package:

Microsoft.Extensions.AI.Evaluation.Reporting v0.9.56-preview

Represents a single execution of a particular iteration of a particular scenario under evaluation.

public ref class ScenarioRun sealed : IAsyncDisposable

public sealed class ScenarioRun : IAsyncDisposable

type ScenarioRun = class
    interface IAsyncDisposable

Public NotInheritable Class ScenarioRun
Implements IAsyncDisposable

Inheritance

Object

ScenarioRun

Implements

IAsyncDisposable

Remarks

Each execution of an evaluation run is assigned a unique ExecutionName. A single such evaluation run can contain evaluations for multiple scenarios each with a unique ScenarioName. The execution of each such scenario in turn can include multiple iterations each with a unique IterationName.

Properties

ChatConfiguration	A ChatConfiguration that specifies the IChatClient and the IEvaluationTokenCounter that are used by AI-based IEvaluators that are invoked as part of the evaluation of this ScenarioRun.
ExecutionName
IterationName	The name of the iteration that this ScenarioRun represents.
ScenarioName	The name of the scenario that this ScenarioRun represents.

Methods

DisposeAsync()	Disposes the ScenarioRun and writes the ScenarioRunResult to the configured IResultStore.
EvaluateAsync(IEnumerable<ChatMessage>, ChatMessage, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied modelResponse and returns an EvaluationResult containing one or more EvaluationMetrics.

Extension Methods

EvaluateAsync(ScenarioRun, ChatMessage, ChatMessage, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied modelResponse and returns an EvaluationResult containing one or more EvaluationMetrics.
EvaluateAsync(ScenarioRun, ChatMessage, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied modelResponse and returns an EvaluationResult containing one or more EvaluationMetrics.
EvaluateAsync(ScenarioRun, String, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied modelResponse and returns an EvaluationResult containing one or more EvaluationMetrics.
EvaluateAsync(ScenarioRun, String, String, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied modelResponse and returns an EvaluationResult containing one or more EvaluationMetrics.