Conversational Analysis Authoring - Get Model Evaluation Results

Gets the detailed results of the evaluation for a trained model. This includes the raw inference results for the data included in the evaluation process.

GET {Endpoint}/language/authoring/analyze-conversations/projects/{projectName}/models/{trainedModelLabel}/evaluation/result?api-version=2023-04-01&stringIndexType=Utf16CodeUnit
GET {Endpoint}/language/authoring/analyze-conversations/projects/{projectName}/models/{trainedModelLabel}/evaluation/result?api-version=2023-04-01&top={top}&skip={skip}&maxpagesize={maxpagesize}&stringIndexType=Utf16CodeUnit

URI Parameters

Name In Required Type Description
Endpoint
path True

string

projectName
path True

string

The new project name.

trainedModelLabel
path True

string

The trained model label.

api-version
query True

string

The API version to use for this operation.

stringIndexType
query True

StringIndexType

Specifies the method used to interpret string offsets. For additional information see https://aka.ms/text-analytics-offsets.

maxpagesize
query

integer

int32

The maximum number of result items per page.

skip
query

integer

int32

The number of result items to skip.

top
query

integer

int32

The number of result items to return.

Responses

Name Type Description
200 OK

PagedConversationalAnalysisAuthoringUtteranceEvaluationResult

The request has succeeded.

Other Status Codes

ErrorResponse

An unexpected error response.

Security

Ocp-Apim-Subscription-Key

Type: apiKey
In: header

OAuth2Auth

Type: oauth2
Flow: accessCode
Authorization URL: https://login.microsoftonline.com/common/oauth2/authorize
Token URL: https://login.microsoftonline.com/common/oauth2/token

Scopes

Name Description
https://cognitiveservices.azure.com/.default

Examples

Successful Get Model Evaluation

Sample request

GET {Endpoint}/language/authoring/analyze-conversations/projects/EmailApp/models/model1/evaluation/result?api-version=2023-04-01&maxpagesize=10&stringIndexType=Utf16CodeUnit

Sample response

{
  "value": [
    {
      "text": "send the email",
      "language": "en-us",
      "entitiesResult": {
        "expectedEntities": [],
        "predictedEntities": []
      },
      "intentsResult": {
        "expectedIntent": "SendEmail",
        "predictedIntent": "SendEmail"
      }
    },
    {
      "text": "send a mail to daniel",
      "language": "en-us",
      "entitiesResult": {
        "expectedEntities": [
          {
            "category": "ContactName",
            "offset": 15,
            "length": 6
          }
        ],
        "predictedEntities": [
          {
            "category": "ContactName",
            "offset": 15,
            "length": 6
          }
        ]
      },
      "intentsResult": {
        "expectedIntent": "SendEmail",
        "predictedIntent": "SendEmail"
      }
    },
    {
      "text": "i forgot to add an important part to that email to james . please set it up to edit",
      "language": "en-us",
      "entitiesResult": {
        "expectedEntities": [
          {
            "category": "ContactName",
            "offset": 51,
            "length": 5
          }
        ],
        "predictedEntities": [
          {
            "category": "Category",
            "offset": 19,
            "length": 9
          },
          {
            "category": "ContactName",
            "offset": 51,
            "length": 5
          }
        ]
      },
      "intentsResult": {
        "expectedIntent": "AddMore",
        "predictedIntent": "AddMore"
      }
    },
    {
      "text": "send email to a and tian",
      "language": "en-us",
      "entitiesResult": {
        "expectedEntities": [
          {
            "category": "ContactName",
            "offset": 14,
            "length": 1
          },
          {
            "category": "ContactName",
            "offset": 20,
            "length": 4
          }
        ],
        "predictedEntities": [
          {
            "category": "ContactName",
            "offset": 14,
            "length": 1
          },
          {
            "category": "ContactName",
            "offset": 20,
            "length": 4
          }
        ]
      },
      "intentsResult": {
        "expectedIntent": "SendEmail",
        "predictedIntent": "SendEmail"
      }
    },
    {
      "text": "send thomas an email",
      "language": "en-us",
      "entitiesResult": {
        "expectedEntities": [
          {
            "category": "ContactName",
            "offset": 5,
            "length": 6
          }
        ],
        "predictedEntities": [
          {
            "category": "ContactName",
            "offset": 5,
            "length": 6
          }
        ]
      },
      "intentsResult": {
        "expectedIntent": "SendEmail",
        "predictedIntent": "SendEmail"
      }
    },
    {
      "text": "i need to add more to the email message i am sending to vincent",
      "language": "en-us",
      "entitiesResult": {
        "expectedEntities": [
          {
            "category": "ContactName",
            "offset": 56,
            "length": 7
          }
        ],
        "predictedEntities": [
          {
            "category": "ContactName",
            "offset": 56,
            "length": 7
          }
        ]
      },
      "intentsResult": {
        "expectedIntent": "AddMore",
        "predictedIntent": "AddMore"
      }
    },
    {
      "text": "send an email to lily roth and abc123@microsoft.com",
      "language": "en-us",
      "entitiesResult": {
        "expectedEntities": [
          {
            "category": "ContactName",
            "offset": 17,
            "length": 9
          }
        ],
        "predictedEntities": [
          {
            "category": "ContactName",
            "offset": 17,
            "length": 9
          }
        ]
      },
      "intentsResult": {
        "expectedIntent": "SendEmail",
        "predictedIntent": "SendEmail"
      }
    },
    {
      "text": "i need to add something else to my email to cheryl",
      "language": "en-us",
      "entitiesResult": {
        "expectedEntities": [
          {
            "category": "ContactName",
            "offset": 44,
            "length": 6
          }
        ],
        "predictedEntities": [
          {
            "category": "ContactName",
            "offset": 44,
            "length": 6
          }
        ]
      },
      "intentsResult": {
        "expectedIntent": "AddMore",
        "predictedIntent": "AddMore"
      }
    },
    {
      "text": "send an email to larry , joseph and billy larkson",
      "language": "en-us",
      "entitiesResult": {
        "expectedEntities": [
          {
            "category": "ContactName",
            "offset": 17,
            "length": 5
          },
          {
            "category": "ContactName",
            "offset": 25,
            "length": 6
          },
          {
            "category": "ContactName",
            "offset": 36,
            "length": 13
          }
        ],
        "predictedEntities": [
          {
            "category": "ContactName",
            "offset": 17,
            "length": 5
          },
          {
            "category": "ContactName",
            "offset": 25,
            "length": 6
          },
          {
            "category": "ContactName",
            "offset": 36,
            "length": 13
          }
        ]
      },
      "intentsResult": {
        "expectedIntent": "SendEmail",
        "predictedIntent": "SendEmail"
      }
    },
    {
      "text": "send mail to dorothy",
      "language": "en-us",
      "entitiesResult": {
        "expectedEntities": [
          {
            "category": "ContactName",
            "offset": 13,
            "length": 7
          }
        ],
        "predictedEntities": [
          {
            "category": "ContactName",
            "offset": 13,
            "length": 7
          }
        ]
      },
      "intentsResult": {
        "expectedIntent": "SendEmail",
        "predictedIntent": "SendEmail"
      }
    }
  ],
  "nextLink": "https://api.example.com/language/authoring/analyze-conversations/projects/EmailApp/models/model1/evaluation/result/?api-version=2023-04-01&top=2147483637&skip=100&maxpagesize=100"
}

Definitions

Name Description
ConversationalAnalysisAuthoringUtteranceEntitiesEvaluationResult

Represents the comparison between the expected and predicted entities for an utterance.

ConversationalAnalysisAuthoringUtteranceEntityEvaluationResult

Represents the evaluation output of an entity label or prediction.

ConversationalAnalysisAuthoringUtteranceEvaluationResult

Represents the evaluation result for an utterance.

ConversationalAnalysisAuthoringUtteranceIntentsEvaluationResult

Represents the comparison between the expected and the predicted intent for an utterance.

Error

The error object.

ErrorCode

Human-readable error code.

ErrorResponse

Error response.

InnerErrorCode

Human-readable error code.

InnerErrorModel

An object containing more specific information about the error. As per Microsoft One API guidelines - https://github.com/Microsoft/api-guidelines/blob/vNext/Guidelines.md#7102-error-condition-responses.

PagedConversationalAnalysisAuthoringUtteranceEvaluationResult

Paged collection of ConversationalAnalysisAuthoringUtteranceEvaluationResult items

StringIndexType

ConversationalAnalysisAuthoringUtteranceEntitiesEvaluationResult

Represents the comparison between the expected and predicted entities for an utterance.

Name Type Description
expectedEntities

ConversationalAnalysisAuthoringUtteranceEntityEvaluationResult[]

Represents the expected entity labels.

predictedEntities

ConversationalAnalysisAuthoringUtteranceEntityEvaluationResult[]

Represents the predicted entity labels.

ConversationalAnalysisAuthoringUtteranceEntityEvaluationResult

Represents the evaluation output of an entity label or prediction.

Name Type Description
category

string

Represents the entity category.

length

integer

Represents the entity length.

offset

integer

Represents the entity offset index relative to the original text.

ConversationalAnalysisAuthoringUtteranceEvaluationResult

Represents the evaluation result for an utterance.

Name Type Description
entitiesResult

ConversationalAnalysisAuthoringUtteranceEntitiesEvaluationResult

Represents the entities results for the utterance.

intentsResult

ConversationalAnalysisAuthoringUtteranceIntentsEvaluationResult

Represents the intents results for the utterance.

language

string

Represents the utterance language. This is BCP-47 representation of a language. For example, use "en" for English, "en-gb" for English (UK), "es" for Spanish etc.

text

string

Represents the utterance text.

ConversationalAnalysisAuthoringUtteranceIntentsEvaluationResult

Represents the comparison between the expected and the predicted intent for an utterance.

Name Type Description
expectedIntent

string

Represents the utterance's expected intent.

predictedIntent

string

Represents the utterance's predicted intent.

Error

The error object.

Name Type Description
code

ErrorCode

One of a server-defined set of error codes.

details

Error[]

An array of details about specific errors that led to this reported error.

innererror

InnerErrorModel

An object containing more specific information than the current object about the error.

message

string

A human-readable representation of the error.

target

string

The target of the error.

ErrorCode

Human-readable error code.

Name Type Description
AzureCognitiveSearchIndexLimitReached

string

AzureCognitiveSearchIndexNotFound

string

AzureCognitiveSearchNotFound

string

AzureCognitiveSearchThrottling

string

Conflict

string

Forbidden

string

InternalServerError

string

InvalidArgument

string

InvalidRequest

string

NotFound

string

OperationNotFound

string

ProjectNotFound

string

QuotaExceeded

string

ServiceUnavailable

string

Timeout

string

TooManyRequests

string

Unauthorized

string

Warning

string

ErrorResponse

Error response.

Name Type Description
error

Error

The error object.

InnerErrorCode

Human-readable error code.

Name Type Description
AzureCognitiveSearchNotFound

string

AzureCognitiveSearchThrottling

string

EmptyRequest

string

ExtractionFailure

string

InvalidCountryHint

string

InvalidDocument

string

InvalidDocumentBatch

string

InvalidParameterValue

string

InvalidRequest

string

InvalidRequestBodyFormat

string

KnowledgeBaseNotFound

string

MissingInputDocuments

string

ModelVersionIncorrect

string

UnsupportedLanguageCode

string

InnerErrorModel

An object containing more specific information about the error. As per Microsoft One API guidelines - https://github.com/Microsoft/api-guidelines/blob/vNext/Guidelines.md#7102-error-condition-responses.

Name Type Description
code

InnerErrorCode

One of a server-defined set of error codes.

details

object

Error details.

innererror

InnerErrorModel

An object containing more specific information than the current object about the error.

message

string

Error message.

target

string

Error target.

PagedConversationalAnalysisAuthoringUtteranceEvaluationResult

Paged collection of ConversationalAnalysisAuthoringUtteranceEvaluationResult items

Name Type Description
nextLink

string

The link to the next page of items

value

ConversationalAnalysisAuthoringUtteranceEvaluationResult[]

The ConversationalAnalysisAuthoringUtteranceEvaluationResult items on this page

StringIndexType

Name Type Description
Utf16CodeUnit

string

The offset and length values will correspond to UTF-16 code units. Use this option if your application is written in a language that support Unicode, for example Java, JavaScript.