Evaluations - List

Reference

Service:: Azure AI Services

API Version:: 3.2

Gets the list of evaluations for the authenticated subscription.

GET {endpoint}/speechtotext/v3.2/evaluations

With optional parameters:

GET {endpoint}/speechtotext/v3.2/evaluations?skip={skip}&top={top}&filter={filter}

URI Parameters

Name	In	Required	Type	Description
endpoint	path	True	string	Supported Cognitive Services endpoints (protocol and hostname, for example: https://westus.api.cognitive.microsoft.com).
filter	query		string	A filtering expression for selecting a subset of the available evaluations. `- Supported properties: displayName, description, createdDateTime, lastActionDateTime, status and locale. - Operators: - eq, ne are supported for all properties. - gt, ge, lt, le are supported for createdDateTime and lastActionDateTime. - and, or, not are supported. - Example: filter=displayName eq 'My evaluation'`
skip	query		integer int32	Number of datasets that will be skipped.
top	query		integer int32	Number of datasets that will be included after skipping.

Responses

Name	Type	Description
200 OK	PaginatedEvaluations	OK Headers Retry-After: integer
Other Status Codes	Error	An error occurred.

Name

Type

Description

200 OK

PaginatedEvaluations

Headers

Retry-After: integer

Other Status Codes

Error

An error occurred.

Security

Ocp-Apim-Subscription-Key

Provide your cognitive services account key here.

Type: apiKey
In: header

Authorization

Provide an access token from the JWT returned by the STS of this region. Make sure to add the management scope to the token by adding the following query string to the STS URL: ?scope=speechservicesmanagement

Type: apiKey
In: header

Examples

Get all evaluations

Get all evaluations with where status is equal to 'Running'

Get all evaluations

Sample request

HTTP

GET {endpoint}/speechtotext/v3.2/evaluations?skip=0&top=2

Sample response

Status code:: 200

{
  "values": [
    {
      "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/evaluations/a55a327b-c866-40b3-a08d-9c0945738633",
      "model1": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/models/base/8a043172-65b7-4cf3-a8b5-4449efa5a0f1"
      },
      "model2": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/models/827712a5-f942-4997-91c3-7c6cde35600b"
      },
      "dataset": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/datasets/9d5f4100-5f8e-4dd6-bd83-9bbbf50d57f1"
      },
      "transcription2": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/transcriptions/ad86d0c9-af6d-4d14-a621-c60d7d65b74f"
      },
      "transcription1": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/transcriptions/ba7ea6f5-3065-40b7-b49a-a90f48584683"
      },
      "links": {
        "files": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/evaluations/00000000-0000-0000-0000-000000000000/files"
      },
      "properties": {
        "wordErrorRate1": 0.065,
        "sentenceErrorRate1": 0.375,
        "tokenErrorRate1": 0.125,
        "sentenceCount1": 8,
        "wordCount1": 46,
        "correctWordCount1": 43,
        "wordSubstitutionCount1": 3,
        "wordDeletionCount1": 0,
        "wordInsertionCount1": 0,
        "tokenCount1": 48,
        "correctTokenCount1": 42,
        "tokenSubstitutionCount1": 6,
        "tokenDeletionCount1": 0,
        "tokenInsertionCount1": 0,
        "tokenErrors1": {
          "punctuation": {
            "numberOfEdits": 2,
            "percentageOfAllEdits": 0.66
          },
          "capitalization": {
            "numberOfEdits": 1,
            "percentageOfAllEdits": 0.33
          }
        },
        "wordErrorRate2": 0.065,
        "sentenceErrorRate2": 0.375,
        "tokenErrorRate2": 0.125,
        "sentenceCount2": 8,
        "wordCount2": 46,
        "correctWordCount2": 43,
        "wordSubstitutionCount2": 3,
        "wordDeletionCount2": 0,
        "wordInsertionCount2": 0,
        "tokenCount2": 48,
        "correctTokenCount2": 42,
        "tokenSubstitutionCount2": 6,
        "tokenDeletionCount2": 0,
        "tokenInsertionCount2": 0,
        "tokenErrors2": {
          "punctuation": {
            "numberOfEdits": 208,
            "percentageOfAllEdits": 1.22
          },
          "capitalization": {
            "numberOfEdits": 126,
            "percentageOfAllEdits": 0.74
          },
          "inverseTextNormalization": {
            "numberOfEdits": 409,
            "percentageOfAllEdits": 2.4
          },
          "lexical": {
            "numberOfEdits": 16284,
            "percentageOfAllEdits": 95.41
          },
          "others": {
            "numberOfEdits": 41,
            "percentageOfAllEdits": 0.24
          }
        }
      },
      "lastActionDateTime": "2019-01-07T11:36:07Z",
      "status": "Succeeded",
      "createdDateTime": "2019-01-07T11:34:12Z",
      "locale": "en-US",
      "displayName": "Evaluation of one model against another model"
    },
    {
      "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/evaluations/daaa45a7-38f0-4918-87d2-bf55fec32ac5",
      "model1": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/models/base/8a043172-65b7-4cf3-a8b5-4449efa5a0f1"
      },
      "model2": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/models/827712a5-f942-4997-91c3-7c6cde35600b"
      },
      "dataset": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/datasets/9d5f4100-5f8e-4dd6-bd83-9bbbf50d57f1"
      },
      "transcription2": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/transcriptions/ad86d0c9-af6d-4d14-a621-c60d7d65b74f"
      },
      "transcription1": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/transcriptions/ba7ea6f5-3065-40b7-b49a-a90f48584683"
      },
      "links": {
        "files": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/evaluations/00000000-0000-0000-0000-000000000000/files"
      },
      "properties": {
        "wordErrorRate1": 0.065,
        "sentenceErrorRate1": 0.375,
        "tokenErrorRate1": 0.125,
        "sentenceCount1": 8,
        "wordCount1": 46,
        "correctWordCount1": 43,
        "wordSubstitutionCount1": 3,
        "wordDeletionCount1": 0,
        "wordInsertionCount1": 0,
        "tokenCount1": 48,
        "correctTokenCount1": 42,
        "tokenSubstitutionCount1": 6,
        "tokenDeletionCount1": 0,
        "tokenInsertionCount1": 0,
        "tokenErrors1": {
          "punctuation": {
            "numberOfEdits": 2,
            "percentageOfAllEdits": 0.66
          },
          "capitalization": {
            "numberOfEdits": 1,
            "percentageOfAllEdits": 0.33
          }
        },
        "wordErrorRate2": 0.065,
        "sentenceErrorRate2": 0.375,
        "tokenErrorRate2": 0.125,
        "sentenceCount2": 8,
        "wordCount2": 46,
        "correctWordCount2": 43,
        "wordSubstitutionCount2": 3,
        "wordDeletionCount2": 0,
        "wordInsertionCount2": 0,
        "tokenCount2": 48,
        "correctTokenCount2": 42,
        "tokenSubstitutionCount2": 6,
        "tokenDeletionCount2": 0,
        "tokenInsertionCount2": 0,
        "tokenErrors2": {
          "punctuation": {
            "numberOfEdits": 208,
            "percentageOfAllEdits": 1.22
          },
          "capitalization": {
            "numberOfEdits": 126,
            "percentageOfAllEdits": 0.74
          },
          "inverseTextNormalization": {
            "numberOfEdits": 409,
            "percentageOfAllEdits": 2.4
          },
          "lexical": {
            "numberOfEdits": 16284,
            "percentageOfAllEdits": 95.41
          },
          "others": {
            "numberOfEdits": 41,
            "percentageOfAllEdits": 0.24
          }
        }
      },
      "lastActionDateTime": "2019-01-07T11:36:07Z",
      "status": "Succeeded",
      "createdDateTime": "2019-01-07T11:34:12Z",
      "locale": "en-US",
      "displayName": "Evaluation of one model against another model"
    }
  ],
  "@nextLink": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/evaluations/1c50ce42-6ab7-454f-8c39-54a752d1a5b6/files?skip=2&top=2"
}

Get all evaluations with where status is equal to 'Running'

Sample request

HTTP

GET {endpoint}/speechtotext/v3.2/evaluations?skip=0&top=2&filter=status%20eq%20'Running'

Sample response

Status code:: 200

{
  "values": [
    {
      "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/evaluations/f85a328c-c866-40b3-a08d-9c0945738633",
      "model1": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/models/base/8a043172-65b7-4cf3-a8b5-4449efa5a0f1"
      },
      "model2": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/models/827712a5-f942-4997-91c3-7c6cde35600b"
      },
      "dataset": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/datasets/9d5f4100-5f8e-4dd6-bd83-9bbbf50d57f1"
      },
      "transcription2": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/transcriptions/ad86d0c9-af6d-4d14-a621-c60d7d65b74f"
      },
      "transcription1": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/transcriptions/ba7ea6f5-3065-40b7-b49a-a90f48584683"
      },
      "links": {
        "files": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/evaluations/00000000-0000-0000-0000-000000000000/files"
      },
      "properties": {
        "wordErrorRate1": 0.89,
        "sentenceErrorRate1": 0.375,
        "tokenErrorRate1": 0.125,
        "sentenceCount1": 8,
        "wordCount1": 46,
        "correctWordCount1": 43,
        "wordSubstitutionCount1": 3,
        "wordDeletionCount1": 0,
        "wordInsertionCount1": 0,
        "tokenCount1": 48,
        "correctTokenCount1": 42,
        "tokenSubstitutionCount1": 6,
        "tokenDeletionCount1": 0,
        "tokenInsertionCount1": 0,
        "wordErrorRate2": 0.98,
        "sentenceErrorRate2": 0.375,
        "tokenErrorRate2": 0.125,
        "sentenceCount2": 8,
        "wordCount2": 46,
        "correctWordCount2": 43,
        "wordSubstitutionCount2": 3,
        "wordDeletionCount2": 0,
        "wordInsertionCount2": 0,
        "tokenCount2": 48,
        "correctTokenCount2": 42,
        "tokenSubstitutionCount2": 6,
        "tokenDeletionCount2": 0,
        "tokenInsertionCount2": 0
      },
      "lastActionDateTime": "2019-01-07T11:36:07Z",
      "status": "Succeeded",
      "createdDateTime": "2019-01-07T11:34:12Z",
      "locale": "en-US",
      "displayName": "Evaluation of one model against another model"
    }
  ]
}

Definitions

Name	Description
DetailedErrorCode	DetailedErrorCode
EditsSummary	EditsSummary
EntityError	EntityError
EntityReference	EntityReference
Error	Error
ErrorCode	ErrorCode
Evaluation	Evaluation
EvaluationLinks	EvaluationLinks
EvaluationProperties	EvaluationProperties
InnerError	InnerError
PaginatedEvaluations	PaginatedEvaluations
Status	Status
TokenErrorDetails	TokenErrorDetails

DetailedErrorCode

Name	Type	Description
DataImportFailed	string	Data import failed.
DeleteNotAllowed	string	Delete not allowed.
DeployNotAllowed	string	Deploy not allowed.
DeployingFailedModel	string	Deploying failed model.
EmptyRequest	string	Empty Request.
EndpointCannotBeDefault	string	Endpoint cannot be default.
EndpointNotUpdatable	string	Endpoint not updatable.
EndpointWithoutLogging	string	Endpoint without logging.
ExceededNumberOfRecordingsUris	string	Exceeded number of recordings uris.
FailedDataset	string	Failed dataset.
Forbidden	string	Forbidden.
InUseViolation	string	In use violation.
InaccessibleCustomerStorage	string	Inaccessible customer storage.
InvalidAdaptationMapping	string	Invalid adaptation mapping.
InvalidBaseModel	string	Invalid base model.
InvalidCallbackUri	string	Invalid callback uri.
InvalidChannels	string	Invalid channels.
InvalidCollection	string	Invalid collection.
InvalidDataset	string	Invalid dataset.
InvalidDocument	string	Invalid Document.
InvalidDocumentBatch	string	Invalid Document Batch.
InvalidLocale	string	Invalid locale.
InvalidLogDate	string	Invalid log date.
InvalidLogEndTime	string	Invalid log end time.
InvalidLogId	string	Invalid log id.
InvalidLogStartTime	string	Invalid log start time.
InvalidModel	string	Invalid model.
InvalidModelUri	string	Invalid model uri.
InvalidParameter	string	Invalid parameter.
InvalidParameterValue	string	Invalid parameter value.
InvalidPayload	string	Invalid payload.
InvalidPermissions	string	Invalid permissions.
InvalidPrerequisite	string	Invalid prerequisite.
InvalidProductId	string	Invalid product id.
InvalidProject	string	Invalid project.
InvalidProjectKind	string	Invalid project kind.
InvalidRecordingsUri	string	Invalid recordings uri.
InvalidRequestBodyFormat	string	Invalid request body format.
InvalidSasValidityDuration	string	Invalid sas validity duration.
InvalidSkipTokenForLogs	string	Invalid skip token for logs.
InvalidSourceAzureResourceId	string	Invalid source Azure resource ID.
InvalidSubscription	string	Invalid subscription.
InvalidTest	string	Invalid test.
InvalidTimeToLive	string	Invalid time to live.
InvalidTopForLogs	string	Invalid top for logs.
InvalidTranscription	string	Invalid transcription.
InvalidWebHookEventKind	string	Invalid web hook event kind.
MissingInputRecords	string	Missing Input Records.
ModelCopyAuthorizationExpired	string	Expired ModelCopyAuthorization.
ModelDeploymentNotCompleteState	string	Model deployment not complete state.
ModelDeprecated	string	Model deprecated.
ModelExists	string	Model exists.
ModelMismatch	string	Model mismatch.
ModelNotDeployable	string	Model not deployable.
ModelVersionIncorrect	string	Model Version Incorrect.
NoUtf8WithBom	string	No utf8 with bom.
OnlyOneOfUrlsOrContainerOrDataset	string	Only one of urls or container or dataset.
ProjectGenderMismatch	string	Project gender mismatch.
QuotaViolation	string	Quota violation.
SingleDefaultEndpoint	string	Single default endpoint.
SkuLimitsExist	string	Sku limits exist.
SubscriptionNotFound	string	Subscription not found.
UnexpectedError	string	Unexpected error.
UnsupportedClassBasedAdaptation	string	Unsupported class based adaptation.
UnsupportedDelta	string	Unsupported delta.
UnsupportedDynamicConfiguration	string	Unsupported dynamic configuration.
UnsupportedFilter	string	Unsupported filter.
UnsupportedLanguageCode	string	Unsupported language code.
UnsupportedOrderBy	string	Unsupported order by.
UnsupportedPagination	string	Unsupported pagination.
UnsupportedTimeRange	string	Unsupported time range.

EditsSummary

Name	Type	Description
numberOfEdits	integer	The optional number of edits for a given type of error of the recognized transcription in comparison with the human transcription.
percentageOfAllEdits	number	The optional percentage of edits for a given type of error of the recognized transcription in comparison with the human transcription.

EntityError

Name	Type	Description
code	string	The code of this error.
message	string	The message for this error.

EntityReference

Name	Type	Description
self	string	The location of the referenced entity.

Error

Name	Type	Description
code	ErrorCode	ErrorCode High level error codes.
details	Error[]	Additional supportive details regarding the error and/or expected policies.
innerError	InnerError	InnerError New Inner Error format which conforms to Cognitive Services API Guidelines which is available at https://microsoft.sharepoint.com/%3Aw%3A/t/CognitiveServicesPMO/EUoytcrjuJdKpeOKIK_QRC8BPtUYQpKBi8JsWyeDMRsWlQ?e=CPq8ow. This contains required properties ErrorCode, message and optional properties target, details(key value pair), inner error(this can be nested).
message	string	High level error message.
target	string	The source of the error. For example it would be "documents" or "document id" in case of invalid document.

ErrorCode

Name	Type	Description
Conflict	string	Representing the conflict error code.
Forbidden	string	Representing the forbidden error code.
InternalCommunicationFailed	string	Representing the internal communication failed error code.
InternalServerError	string	Representing the internal server error error code.
InvalidArgument	string	Representing the invalid argument error code.
InvalidRequest	string	Representing the invalid request error code.
NotAllowed	string	Representing the not allowed error code.
NotFound	string	Representing the not found error code.
PipelineError	string	Representing the pipeline error error code.
ServiceUnavailable	string	Representing the service unavailable error code.
TooManyRequests	string	Representing the too many requests error code.
Unauthorized	string	Representing the unauthorized error code.
UnprocessableEntity	string	Representing the unprocessable entity error code.
UnsupportedMediaType	string	Representing the unsupported media type error code.

Evaluation

Name	Type	Description
createdDateTime	string	The time-stamp when the object was created. The time stamp is encoded as ISO 8601 date and time format ("YYYY-MM-DDThh:mm:ssZ", see https://en.wikipedia.org/wiki/ISO_8601#Combined_date_and_time_representations).
customProperties	object	The custom properties of this entity. The maximum allowed key length is 64 characters, the maximum allowed value length is 256 characters and the count of allowed entries is 10.
dataset	EntityReference	EntityReference
description	string	The description of the object.
displayName	string	The display name of the object.
lastActionDateTime	string	The time-stamp when the current status was entered. The time stamp is encoded as ISO 8601 date and time format ("YYYY-MM-DDThh:mm:ssZ", see https://en.wikipedia.org/wiki/ISO_8601#Combined_date_and_time_representations).
links	EvaluationLinks	EvaluationLinks
locale	string	The locale of the contained data.
model1	EntityReference	EntityReference
model2	EntityReference	EntityReference
project	EntityReference	EntityReference
properties	EvaluationProperties	EvaluationProperties
self	string	The location of this entity.
status	Status	Status Describe the current state of the API.
transcription1	EntityReference	EntityReference
transcription2	EntityReference	EntityReference

EvaluationLinks

Name	Type	Description
files	string	The location to get all files of this entity. See operation "Evaluations_ListFiles" for more details.

EvaluationProperties

Name	Type	Description
correctTokenCount1	integer	The optional number of correctly recognized tokens by model1.
correctTokenCount2	integer	The optional number of correctly recognized tokens by model2.
correctWordCount1	integer	The number of correctly recognized words by model1.
correctWordCount2	integer	The number of correctly recognized words by model2.
email	string	The email address to send email notifications to in case the operation completes. The value will be removed after successfully sending the email.
error	EntityError	EntityError
sentenceCount1	integer	The number of processed sentences by model1.
sentenceCount2	integer	The number of processed sentences by model2.
sentenceErrorRate1	number	The sentence error rate of recognition with model1.
sentenceErrorRate2	number	The sentence error rate of recognition with model2.
tokenCount1	integer	Optional the number of processed tokens by model1.
tokenCount2	integer	The optional number of processed tokens by model2.
tokenDeletionCount1	integer	The optional number of recognized tokens by model1, that are deletions.
tokenDeletionCount2	integer	The optional number of recognized tokens by model2, that are deletions.
tokenErrorRate1	number	The optional token error rate of recognition with model1.
tokenErrorRate2	number	The optional token error rate of recognition with model2.
tokenErrors1	TokenErrorDetails	TokenErrorDetails
tokenErrors2	TokenErrorDetails	TokenErrorDetails
tokenInsertionCount1	integer	The optional number of recognized tokens by model1, that are insertions.
tokenInsertionCount2	integer	The optional number of recognized tokens by model2, that are insertions.
tokenSubstitutionCount1	integer	The optional number of recognized tokens by model1, that are substitutions.
tokenSubstitutionCount2	integer	The optional number of recognized tokens by model2, that are substitutions.
wordCount1	integer	The number of processed words by model1.
wordCount2	integer	The number of processed words by model2.
wordDeletionCount1	integer	The number of recognized words by model1, that are deletions.
wordDeletionCount2	integer	The number of recognized words by model2, that are deletions.
wordErrorRate1	number	The word error rate of recognition with model1.
wordErrorRate2	number	The word error rate of recognition with model2.
wordInsertionCount1	integer	The number of recognized words by model1, that are insertions.
wordInsertionCount2	integer	The number of recognized words by model2, that are insertions.
wordSubstitutionCount1	integer	The number of recognized words by model1, that are substitutions.
wordSubstitutionCount2	integer	The number of recognized words by model2, that are substitutions.

InnerError

Name	Type	Description
code	DetailedErrorCode	DetailedErrorCode Detailed error code enum.
details	object	Additional supportive details regarding the error and/or expected policies.
innerError	InnerError	InnerError New Inner Error format which conforms to Cognitive Services API Guidelines which is available at https://microsoft.sharepoint.com/%3Aw%3A/t/CognitiveServicesPMO/EUoytcrjuJdKpeOKIK_QRC8BPtUYQpKBi8JsWyeDMRsWlQ?e=CPq8ow. This contains required properties ErrorCode, message and optional properties target, details(key value pair), inner error(this can be nested).
message	string	High level error message.
target	string	The source of the error. For example it would be "documents" or "document id" in case of invalid document.

PaginatedEvaluations

Name	Type	Description
@nextLink	string	A link to the next set of paginated results if there are more entities available; otherwise null.
values	Evaluation[]	A list of entities limited by either the passed query parameters 'skip' and 'top' or their default values. When iterating through a list using pagination and deleting entities in parallel, some entities will be skipped in the results. It's recommended to build a list on the client and delete after the fetching of the complete list.

Name

Type

Description

@nextLink

string

A link to the next set of paginated results if there are more entities available; otherwise null.

values

Evaluation[]

A list of entities limited by either the passed query parameters 'skip' and 'top' or their default values.

When iterating through a list using pagination and deleting entities in parallel, some entities will be skipped in the results. It's recommended to build a list on the client and delete after the fetching of the complete list.

Status

Name	Type	Description
Failed	string	The long running operation has failed.
NotStarted	string	The long running operation has not yet started.
Running	string	The long running operation is currently processing.
Succeeded	string	The long running operation has successfully completed.

TokenErrorDetails

Name	Type	Description
capitalization	EditsSummary	EditsSummary
inverseTextNormalization	EditsSummary	EditsSummary
lexical	EditsSummary	EditsSummary
others	EditsSummary	EditsSummary
punctuation	EditsSummary	EditsSummary

Share via

Evaluations - List

URI Parameters

Responses

Security

Ocp-Apim-Subscription-Key

Authorization

Examples

Get all evaluations

Sample request

Sample response

Get all evaluations with where status is equal to 'Running'

Sample request

Sample response

Definitions

DetailedErrorCode

EditsSummary

EntityError

EntityReference

Error

ErrorCode

Evaluation

EvaluationLinks

EvaluationProperties

InnerError

PaginatedEvaluations

Status

TokenErrorDetails

Additional resources