Muokkaa

Jaa

AML skill in an Azure AI Search enrichment pipeline

  • Artikkeli

Important

Support for indexer connections to the Azure AI Foundry model catalog is in public preview under supplemental terms of use. Preview REST APIs support this skill.

The AML skill allows you to extend AI enrichment with a custom Azure Machine Learning (AML) model or deployed base embedding model on Azure AI Foundry. Once an AML model is trained and deployed, an AML skill integrates it into a skillset.

AML skill usage

Like other built-in skills, a custom AML skill has inputs and outputs. The inputs are sent to a deployed AML online endpoint as a JSON object. The output of the endpoint must be a JSON payload in the response, along with a success status code. Your data is processed in the Geo where your model is deployed. The response is expected to provide the outputs specified by your AML skill definition. Any other response is considered an error and no enrichments are performed.

Note

The indexer will retry twice for certain standard HTTP status codes returned from the AML online endpoint. These HTTP status codes are:

  • 503 Service Unavailable
  • 429 Too Many Requests

The AML skill can be called with the 2024-07-01 stable API version or equivalent Azure SDK, or the 2024-05-01-preview API version for connections to the model catalog in Azure AI Foundry portal.

AML skill for models in Azure AI Foundry

Starting in 2024-05-01-preview REST API and in the Azure portal (which also targets the 2024-05-01-preview), Azure AI Search provides the Azure AI Foundry model catalog vectorizer for query time connections to the model catalog in Azure AI Foundry portal. If you want to use that vectorizer for queries, an AML skill is the indexing counterpart for generating embeddings using a model in the Azure AI Foundry model catalog.

During indexing, the AML skill can connect to the model catalog to generate vectors for the index. At query time, queries can use a vectorizer to connect to the same model to vectorize text strings for a vector query. In this workflow, the AML skill and the model catalog vectorizer should be used together so that you're using the same embedding model for both indexing and queries. See Use embedding models from Azure AI Foundry model catalog for details and for a list of the supported embedding models.

We recommend using the Import and vectorize data wizard to generate a skillset that includes an AML skill for deployed embedding models on Azure AI Foundry. AML skill definition for inputs, outputs, and mappings are generated by the wizard, which gives you an easy way to test a model before writing any code.

Prerequisites

  • An AML workspace for a custom model that you create, or a project in Azure AI Foundry if an embedding model is deployed from the catalog.
  • An Online endpoints (real-time) in this workspace for a custom model, or the model endpoint for embedding models deployed from the catalog.

@odata.type

Microsoft.Skills.Custom.AmlSkill

Skill parameters

Parameters are case-sensitive. Which parameters you choose to use depends on what authentication your AML online endpoint requires, if any

Parameter name Description
uri (Required for key authentication) The scoring URI of the AML online endpoint to which the JSON payload is sent. Only the https URI scheme is allowed. For embedding models in the Azure AI Foundry model catalog, this is the target URI.
key (Required for key authentication) The key for the AML online endpoint or the
resourceId (Required for token authentication). The Azure Resource Manager resource ID of the AML online endpoint. It should be in the format subscriptions/{guid}/resourceGroups/{resource-group-name}/Microsoft.MachineLearningServices/workspaces/{workspace-name}/onlineendpoints/{endpoint_name}.
region (Optional for token authentication). The region the AML online endpoint is deployed in.
timeout (Optional) When specified, indicates the timeout for the http client making the API call. It must be formatted as an XSD "dayTimeDuration" value (a restricted subset of an ISO 8601 duration value). For example, PT60S for 60 seconds. If not set, a default value of 30 seconds is chosen. The timeout can be set to a maximum of 230 seconds and a minimum of 1 second.
degreeOfParallelism (Optional) When specified, indicates the number of calls the indexer makes in parallel to the endpoint you have provided. You can decrease this value if your endpoint is failing under too high of a request load. You can raise it if your endpoint is able to accept more requests and you would like an increase in the performance of the indexer. If not set, a default value of 5 is used. The degreeOfParallelism can be set to a maximum of 10 and a minimum of 1.

Authentication

AML online endpoints provide two authentication options:

  • Key-based authentication. A static key is provided to authenticate scoring requests from AML skills. Set the uri and key parameters for this connection.

  • Token-Based Authentication, where the AML online endpoint is deployed using token based authentication. The Azure AI Search service's managed identity must be enabled and have a role assignment on workspace. The AML skill then uses the service's managed identity to authenticate against the AML online endpoint, with no static keys required. The search service identity must be an Owner or Contributor. Set the resourceId parameter, and if the search service is in a different region from the AML workspace, set the region parameter.

Skill inputs

Skill inputs are a node of the enriched document that's created during document cracking. For example, it might be the root document, a normalized image, or the content of a blob. There are no predefined inputs for this skill. For inputs, you should specify one or more nodes that are populated at the time of the AML skill's execution.

Skill outputs

Skill outputs are new nodes of an enriched document created by the skill. There are no predefined outputs for this skill. For outputs, you should provide nodes that can be populated from the JSON response of your AML skill.

Sample definition

  {
    "@odata.type": "#Microsoft.Skills.Custom.AmlSkill",
    "description": "A custom model that detects the language in a document.",
    "uri": "https://language-model.models.contoso.com/score",
    "context": "/document",
    "inputs": [
      {
        "name": "text",
        "source": "/document/content"
      }
    ],
    "outputs": [
      {
        "name": "detected_language_code"
      }
    ]
  }

Sample input JSON structure

This JSON structure represents the payload that is sent to your AML online endpoint. The top-level fields of the structure correspond to the "names" specified in the inputs section of the skill definition. The values of those fields are from the source of those fields (which could be from a field in the document, or potentially from another skill)

{
  "text": "Este es un contrato en Inglés"
}

Sample output JSON structure

The output corresponds to the response returned from your AML online endpoint. The AML online endpoint should only return a JSON payload (verified by looking at the Content-Type response header) and should be an object where the fields are enrichments matching the "names" in the output and whose value is considered the enrichment.

{
    "detected_language_code": "es"
}

Inline shaping sample definition

  {
    "@odata.type": "#Microsoft.Skills.Custom.AmlSkill",
    "description": "A sample model that detects the language of sentence",
    "uri": "https://language-model.models.contoso.com/score",
    "context": "/document",
    "inputs": [
      {
        "name": "shapedText",
        "sourceContext": "/document",
        "inputs": [
            {
              "name": "content",
              "source": "/document/content"
            }
        ]
      }
    ],
    "outputs": [
      {
        "name": "detected_language_code"
      }
    ]
  }

Inline shaping input JSON structure

{
  "shapedText": { "content": "Este es un contrato en Inglés" }
}

Inline shaping sample output JSON structure

{
    "detected_language_code": "es"
}

Error cases

In addition to your AML being unavailable or sending out nonsuccessful status codes, the following are considered erroneous cases:

  • The AML online endpoint returns a success status code, but the response indicates that it isn't application/json, then the response is considered invalid and no enrichments are performed.

  • The AML online endpoint returns invalid JSON.

For cases when the AML online endpoint is unavailable or returns an HTTP error, a friendly error with any available details about the HTTP error is added to the indexer execution history.

See also