Share via

Evaluations - List

  • Reference
Service:
Azure AI Services
API Version:
v3.2-preview.2

Gets the list of evaluations for the authenticated subscription.

GET {endpoint}/speechtotext/v3.2-preview.2/evaluations
With optional parameters: 
GET {endpoint}/speechtotext/v3.2-preview.2/evaluations?skip={skip}&top={top}&filter={filter}

URI Parameters

Name In Required Type Description
endpoint
 path True

string

Supported Cognitive Services endpoints (protocol and hostname, for example: https://westus.api.cognitive.microsoft.com).
filter
 query

string

A filtering expression for selecting a subset of the available evaluations.

        - Supported properties: displayName, description, createdDateTime, lastActionDateTime, status and locale.

        - Operators:

          - eq, ne are supported for all properties.

          - gt, ge, lt, le are supported for createdDateTime and lastActionDateTime.

          - and, or, not are supported.

        - Example:

          filter=displayName eq 'My evaluation'
skip
 query

integer

int32

Number of datasets that will be skipped.
top
 query

integer

int32

Number of datasets that will be included after skipping.

Responses

Name Type Description
200 OK

PaginatedEvaluations

OK

Headers

Retry-After: integer
Other Status Codes

Error

An error occurred.

Security

Ocp-Apim-Subscription-Key

Provide your cognitive services account key here.

Type: apiKey
In: header

Authorization

Provide an access token from the JWT returned by the STS of this region. Make sure to add the management scope to the token by adding the following query string to the STS URL: ?scope=speechservicesmanagement

Type: apiKey
In: header

Examples

Get all evaluations
Get all evaluations with where status is equal to 'Running'

Get all evaluations

Sample request

GET {endpoint}/speechtotext/v3.2-preview.2/evaluations?skip=0&top=2

Sample response

Status code:
200 
{
  "values": [
    {
      "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2-preview.2/evaluations/a55a327b-c866-40b3-a08d-9c0945738633",
      "model1": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2-preview.2/models/base/8a043172-65b7-4cf3-a8b5-4449efa5a0f1"
      },
      "model2": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2-preview.2/models/827712a5-f942-4997-91c3-7c6cde35600b"
      },
      "dataset": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2-preview.2/datasets/9d5f4100-5f8e-4dd6-bd83-9bbbf50d57f1"
      },
      "transcription2": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2-preview.2/transcriptions/ad86d0c9-af6d-4d14-a621-c60d7d65b74f"
      },
      "transcription1": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2-preview.2/transcriptions/ba7ea6f5-3065-40b7-b49a-a90f48584683"
      },
      "links": {
        "files": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2-preview.2/evaluations/00000000-0000-0000-0000-000000000000/files"
      },
      "properties": {
        "wordErrorRate2": 0.065,
        "wordErrorRate1": 0.065,
        "sentenceErrorRate2": 0.375,
        "sentenceCount2": 8,
        "wordCount2": 46,
        "correctWordCount2": 43,
        "wordSubstitutionCount2": 3,
        "wordDeletionCount2": 0,
        "wordInsertionCount2": 0,
        "sentenceErrorRate1": 0.375,
        "sentenceCount1": 8,
        "wordCount1": 46,
        "correctWordCount1": 43,
        "wordSubstitutionCount1": 3,
        "wordDeletionCount1": 0,
        "wordInsertionCount1": 0,
        "tokenErrorRate2": 0.125,
        "tokenErrorRate1": 0.125,
        "tokenCount2": 48,
        "correctTokenCount2": 42,
        "tokenSubstitutionCount2": 6,
        "tokenDeletionCount2": 0,
        "tokenInsertionCount2": 0,
        "tokenCount1": 48,
        "correctTokenCount1": 42,
        "tokenSubstitutionCount1": 6,
        "tokenDeletionCount1": 0,
        "tokenInsertionCount1": 0
      },
      "lastActionDateTime": "2019-01-07T11:36:07Z",
      "status": "Succeeded",
      "createdDateTime": "2019-01-07T11:34:12Z",
      "locale": "en-US",
      "displayName": "Evalution of one model against another model"
    },
    {
      "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2-preview.2/evaluations/daaa45a7-38f0-4918-87d2-bf55fec32ac5",
      "model1": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2-preview.2/models/base/8a043172-65b7-4cf3-a8b5-4449efa5a0f1"
      },
      "model2": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2-preview.2/models/827712a5-f942-4997-91c3-7c6cde35600b"
      },
      "dataset": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2-preview.2/datasets/9d5f4100-5f8e-4dd6-bd83-9bbbf50d57f1"
      },
      "transcription2": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2-preview.2/transcriptions/ad86d0c9-af6d-4d14-a621-c60d7d65b74f"
      },
      "transcription1": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2-preview.2/transcriptions/ba7ea6f5-3065-40b7-b49a-a90f48584683"
      },
      "links": {
        "files": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2-preview.2/evaluations/00000000-0000-0000-0000-000000000000/files"
      },
      "properties": {
        "wordErrorRate2": 0.065,
        "wordErrorRate1": 0.065,
        "sentenceErrorRate2": 0.375,
        "sentenceCount2": 8,
        "wordCount2": 46,
        "correctWordCount2": 43,
        "wordSubstitutionCount2": 3,
        "wordDeletionCount2": 0,
        "wordInsertionCount2": 0,
        "sentenceErrorRate1": 0.375,
        "sentenceCount1": 8,
        "wordCount1": 46,
        "correctWordCount1": 43,
        "wordSubstitutionCount1": 3,
        "wordDeletionCount1": 0,
        "wordInsertionCount1": 0,
        "tokenErrorRate2": 0.125,
        "tokenErrorRate1": 0.125,
        "tokenCount2": 48,
        "correctTokenCount2": 42,
        "tokenSubstitutionCount2": 6,
        "tokenDeletionCount2": 0,
        "tokenInsertionCount2": 0,
        "tokenCount1": 48,
        "correctTokenCount1": 42,
        "tokenSubstitutionCount1": 6,
        "tokenDeletionCount1": 0,
        "tokenInsertionCount1": 0
      },
      "lastActionDateTime": "2019-01-07T11:36:07Z",
      "status": "Succeeded",
      "createdDateTime": "2019-01-07T11:34:12Z",
      "locale": "en-US",
      "displayName": "Evalution of one model against another model"
    }
  ],
  "@nextLink": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2-preview.2/evaluations/1c50ce42-6ab7-454f-8c39-54a752d1a5b6/files?skip=2&top=2"
}

Get all evaluations with where status is equal to 'Running'

Sample request

GET {endpoint}/speechtotext/v3.2-preview.2/evaluations?skip=0&top=2&filter=status%20eq%20'Running'

Sample response

Status code:
200 
{
  "values": [
    {
      "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2-preview.2/evaluations/f85a328c-c866-40b3-a08d-9c0945738633",
      "model1": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2-preview.2/models/base/8a043172-65b7-4cf3-a8b5-4449efa5a0f1"
      },
      "model2": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2-preview.2/models/827712a5-f942-4997-91c3-7c6cde35600b"
      },
      "dataset": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2-preview.2/datasets/9d5f4100-5f8e-4dd6-bd83-9bbbf50d57f1"
      },
      "transcription2": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2-preview.2/transcriptions/ad86d0c9-af6d-4d14-a621-c60d7d65b74f"
      },
      "transcription1": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2-preview.2/transcriptions/ba7ea6f5-3065-40b7-b49a-a90f48584683"
      },
      "links": {
        "files": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2-preview.2/evaluations/00000000-0000-0000-0000-000000000000/files"
      },
      "properties": {
        "wordErrorRate2": 0.98,
        "wordErrorRate1": 0.89,
        "sentenceErrorRate2": 0.375,
        "sentenceCount2": 8,
        "wordCount2": 46,
        "correctWordCount2": 43,
        "wordSubstitutionCount2": 3,
        "wordDeletionCount2": 0,
        "wordInsertionCount2": 0,
        "sentenceErrorRate1": 0.375,
        "sentenceCount1": 8,
        "wordCount1": 46,
        "correctWordCount1": 43,
        "wordSubstitutionCount1": 3,
        "wordDeletionCount1": 0,
        "wordInsertionCount1": 0,
        "tokenErrorRate2": 0.125,
        "tokenErrorRate1": 0.125,
        "tokenCount2": 48,
        "correctTokenCount2": 42,
        "tokenSubstitutionCount2": 6,
        "tokenDeletionCount2": 0,
        "tokenInsertionCount2": 0,
        "tokenCount1": 48,
        "correctTokenCount1": 42,
        "tokenSubstitutionCount1": 6,
        "tokenDeletionCount1": 0,
        "tokenInsertionCount1": 0
      },
      "lastActionDateTime": "2019-01-07T11:36:07Z",
      "status": "Succeeded",
      "createdDateTime": "2019-01-07T11:34:12Z",
      "locale": "en-US",
      "displayName": "Evalution of one model against another model"
    }
  ]
}

Definitions

Name Description
DetailedErrorCode

DetailedErrorCode
EntityError

EntityError
EntityReference

EntityReference
Error

Error
ErrorCode

ErrorCode
Evaluation

Evaluation
EvaluationLinks

EvaluationLinks
EvaluationProperties

EvaluationProperties
InnerError

InnerError
PaginatedEvaluations

PaginatedEvaluations
Status

Status

DetailedErrorCode

Enumeration

DetailedErrorCode

Value Description
DataImportFailed

Data import failed.
DeleteNotAllowed

Delete not allowed.
DeployNotAllowed

Deploy not allowed.
DeployingFailedModel

Deploying failed model.
EmptyRequest

Empty Request.
EndpointCannotBeDefault

Endpoint cannot be default.
EndpointNotUpdatable

Endpoint not updatable.
EndpointWithoutLogging

Endpoint without logging.
ExceededNumberOfRecordingsUris

Exceeded number of recordings uris.
FailedDataset

Failed dataset.
Forbidden

Forbidden.
InUseViolation

In use violation.
InaccessibleCustomerStorage

Inaccessible customer storage.
InvalidAdaptationMapping

Invalid adaptation mapping.
InvalidBaseModel

Invalid base model.
InvalidCallbackUri

Invalid callback uri.
InvalidCollection

Invalid collection.
InvalidDataset

Invalid dataset.
InvalidDocument

Invalid Document.
InvalidDocumentBatch

Invalid Document Batch.
InvalidLocale

Invalid locale.
InvalidLogDate

Invalid log date.
InvalidLogEndTime

Invalid log end time.
InvalidLogId

Invalid log id.
InvalidLogStartTime

Invalid log start time.
InvalidModel

Invalid model.
InvalidModelUri

Invalid model uri.
InvalidParameter

Invalid parameter.
InvalidParameterValue

Invalid parameter value.
InvalidPayload

Invalid payload.
InvalidPermissions

Invalid permissions.
InvalidPrerequisite

Invalid prerequisite.
InvalidProductId

Invalid product id.
InvalidProject

Invalid project.
InvalidProjectKind

Invalid project kind.
InvalidRecordingsUri

Invalid recordings uri.
InvalidRequestBodyFormat

Invalid request body format.
InvalidSasValidityDuration

Invalid sas validity duration.
InvalidSkipTokenForLogs

Invalid skip token for logs.
InvalidSourceAzureResourceId

Invalid source Azure resource ID.
InvalidSubscription

Invalid subscription.
InvalidTest

Invalid test.
InvalidTimeToLive

Invalid time to live.
InvalidTopForLogs

Invalid top for logs.
InvalidTranscription

Invalid transcription.
InvalidWebHookEventKind

Invalid web hook event kind.
MissingInputRecords

Missing Input Records.
ModelCopyOperationExists

Model copy operation exists.
ModelDeploymentNotCompleteState

Model deployment not complete state.
ModelDeprecated

Model deprecated.
ModelExists

Model exists.
ModelMismatch

Model mismatch.
ModelNotDeployable

Model not deployable.
ModelVersionIncorrect

Model Version Incorrect.
NoUtf8WithBom

No utf8 with bom.
OnlyOneOfUrlsOrContainerOrDataset

Only one of urls or container or dataset.
ProjectGenderMismatch

Project gender mismatch.
QuotaViolation

Quota violation.
SingleDefaultEndpoint

Single default endpoint.
SkuLimitsExist

Sku limits exist.
SubscriptionNotFound

Subscription not found.
UnexpectedError

Unexpected error.
UnsupportedClassBasedAdaptation

Unsupported class based adaptation.
UnsupportedDelta

Unsupported delta.
UnsupportedDynamicConfiguration

Unsupported dynamic configuration.
UnsupportedFilter

Unsupported filter.
UnsupportedLanguageCode

Unsupported language code.
UnsupportedOrderBy

Unsupported order by.
UnsupportedPagination

Unsupported pagination.
UnsupportedTimeRange

Unsupported time range.

EntityError

Object

EntityError

Name Type Description
code

string

The code of this error.
message

string

The message for this error.

EntityReference

Object

EntityReference

Name Type Description
self

string

The location of the referenced entity.

Error

Object

Error

Name Type Description
code

ErrorCode

ErrorCode
High level error codes.
details

Error[]

Additional supportive details regarding the error and/or expected policies.
innerError

InnerError

InnerError
New Inner Error format which conforms to Cognitive Services API Guidelines which is available at https://microsoft.sharepoint.com/%3Aw%3A/t/CognitiveServicesPMO/EUoytcrjuJdKpeOKIK_QRC8BPtUYQpKBi8JsWyeDMRsWlQ?e=CPq8ow. This contains required properties ErrorCode, message and optional properties target, details(key value pair), inner error(this can be nested).
message

string

High level error message.
target

string

The source of the error. For example it would be "documents" or "document id" in case of invalid document.

ErrorCode

Enumeration

ErrorCode

Value Description
Conflict

Representing the conflict error code.
Forbidden

Representing the forbidden error code.
InternalCommunicationFailed

Representing the internal communication failed error code.
InternalServerError

Representing the internal server error error code.
InvalidArgument

Representing the invalid argument error code.
InvalidRequest

Representing the invalid request error code.
NotAllowed

Representing the not allowed error code.
NotFound

Representing the not found error code.
PipelineError

Representing the pipeline error error code.
ServiceUnavailable

Representing the service unavailable error code.
TooManyRequests

Representing the too many requests error code.
Unauthorized

Representing the unauthorized error code.
UnprocessableEntity

Representing the unprocessable entity error code.
UnsupportedMediaType

Representing the unsupported media type error code.

Evaluation

Object

Evaluation

Name Type Description
createdDateTime

string

The time-stamp when the object was created. The time stamp is encoded as ISO 8601 date and time format ("YYYY-MM-DDThh:mm:ssZ", see https://en.wikipedia.org/wiki/ISO_8601#Combined_date_and_time_representations).
customProperties

object

The custom properties of this entity. The maximum allowed key length is 64 characters, the maximum allowed value length is 256 characters and the count of allowed entries is 10.
dataset

EntityReference

EntityReference
description

string

The description of the object.
displayName

string

The display name of the object.
lastActionDateTime

string

The time-stamp when the current status was entered. The time stamp is encoded as ISO 8601 date and time format ("YYYY-MM-DDThh:mm:ssZ", see https://en.wikipedia.org/wiki/ISO_8601#Combined_date_and_time_representations).
links

EvaluationLinks

EvaluationLinks
locale

string

The locale of the contained data.
model1

EntityReference

EntityReference
model2

EntityReference

EntityReference
project

EntityReference

EntityReference
properties

EvaluationProperties

EvaluationProperties
self

string

The location of this entity.
status

Status

Status
Describe the current state of the API
transcription1

EntityReference

EntityReference
transcription2

EntityReference

EntityReference
Object

EvaluationLinks

Name Type Description
files

string

The location to get all files of this entity. See operation "Evaluations_ListFiles" for more details.

EvaluationProperties

Object

EvaluationProperties

Name Type Description
correctTokenCount1

integer

The number of correctly recognized tokens by model1.
correctTokenCount2

integer

The number of correctly recognized tokens by model2.
correctWordCount1

integer

The number of correctly recognized words by model1.
correctWordCount2

integer

The number of correctly recognized words by model2.
email

string

The email address to send email notifications to in case the operation completes. The value will be removed after successfully sending the email.
error

EntityError

EntityError
sentenceCount1

integer

The number of processed sentences by model1.
sentenceCount2

integer

The number of processed sentences by model2.
sentenceErrorRate1

number

The sentence error rate of recognition with model1.
sentenceErrorRate2

number

The sentence error rate of recognition with model2.
tokenCount1

integer

The number of processed tokens by model1.
tokenCount2

integer

The number of processed tokens by model2.
tokenDeletionCount1

integer

The number of recognized tokens by model1, that are deletions.
tokenDeletionCount2

integer

The number of recognized tokens by model2, that are deletions.
tokenErrorRate1

number

The token error rate of recognition with model1.
tokenErrorRate2

number

The token error rate of recognition with model2.
tokenInsertionCount1

integer

The number of recognized tokens by model1, that are insertions.
tokenInsertionCount2

integer

The number of recognized tokens by model2, that are insertions.
tokenSubstitutionCount1

integer

The number of recognized tokens by model1, that are substitutions.
tokenSubstitutionCount2

integer

The number of recognized tokens by model2, that are substitutions.
wordCount1

integer

The number of processed words by model1.
wordCount2

integer

The number of processed words by model2.
wordDeletionCount1

integer

The number of recognized words by model1, that are deletions.
wordDeletionCount2

integer

The number of recognized words by model2, that are deletions.
wordErrorRate1

number

The word error rate of recognition with model1.
wordErrorRate2

number

The word error rate of recognition with model2.
wordInsertionCount1

integer

The number of recognized words by model1, that are insertions.
wordInsertionCount2

integer

The number of recognized words by model2, that are insertions.
wordSubstitutionCount1

integer

The number of recognized words by model1, that are substitutions.
wordSubstitutionCount2

integer

The number of recognized words by model2, that are substitutions.

InnerError

Object

InnerError

Name Type Description
code

DetailedErrorCode

DetailedErrorCode
Detailed error code enum.
details

object

Additional supportive details regarding the error and/or expected policies.
innerError

InnerError

InnerError
New Inner Error format which conforms to Cognitive Services API Guidelines which is available at https://microsoft.sharepoint.com/%3Aw%3A/t/CognitiveServicesPMO/EUoytcrjuJdKpeOKIK_QRC8BPtUYQpKBi8JsWyeDMRsWlQ?e=CPq8ow. This contains required properties ErrorCode, message and optional properties target, details(key value pair), inner error(this can be nested).
message

string

High level error message.
target

string

The source of the error. For example it would be "documents" or "document id" in case of invalid document.

PaginatedEvaluations

Object

PaginatedEvaluations

Name Type Description
@nextLink

string

A link to the next set of paginated results if there are more entities available; otherwise null.
values

Evaluation[]

A list of entities limited by either the passed query parameters 'skip' and 'top' or their default values.

When iterating through a list using pagination and deleting entities in parallel, some entities will be skipped in the results. It's recommended to build a list on the client and delete after the fetching of the complete list.

Status

Enumeration

Status

Value Description
Failed

The long running operation has failed.
NotStarted

The long running operation has not yet started.
Running

The long running operation is currently processing.
Succeeded

The long running operation has successfully completed.