Attach an Azure AI multi-service resource to a skillset in Azure AI Search

Article
11/19/2024

When configuring an optional AI enrichment pipeline in Azure AI Search, you can enrich a small number of documents free of charge, limited to 20 transactions daily per index. For larger and more frequent workloads, you should attach a billable Azure AI multi-service resource.

A multi-service account provides a collection of Azure AI services, rather than individual services. Providing an account in an Azure AI Search skillset allows Microsoft to charge you for using these services:

Azure AI Vision for image analysis, optical character recognition (OCR), and multimodal text and image embedding.
Azure AI Language for language detection, entity recognition, sentiment analysis, and key phrase extraction
Azure AI Speech for speech to text and text to speech
Azure AI Translator for machine text translation

You must provide connection information to the Azure AI multi-resource in the skillset. Azure AI Search doesn't use the connection for skillset workloads, but it does use the connection to access the billing meters on the resource. As such, your Azure AI services account is used for billing, not skills processing. Azure AI Search uses separate dedicated resources for skills processing.

You can use a key on the connection, or implement a keyless approach that's currently in preview.

Tip

Azure provides infrastructure for you to monitor billing and budgets. For more information about monitoring Azure AI services, see Plan and manage costs for Azure AI services.

Prerequisites

Connectivity over a public endpoint, unless your search service meets the creation date, tier, and region requirements for private connections to an Azure AI multi-service account.

Note

If your Azure AI resource is configured to use a private endpoint, Azure AI Search can connect using a shared private link if the search service was created after April 3, 2024 and is in a region that provides higher capacity computing power and is on a supported tier. For more information, see the requirements for using shared private links.

Bill through a keyless connection

Note

This feature is currently in public preview. This preview is provided without a service-level agreement, and is not recommended for production workloads. Certain features might not be supported or might have constrained capabilities. For more information, see Supplemental Terms of Use for Microsoft Azure Previews.

Using the Azure portal or newer preview REST APIs and beta SDK packages, you can attach an Azure AI multi-service resource using a managed identity and permissions. The advantage of this approach is that billing is keyless and has no dependency on regions.

Configure Azure AI Search to use a managed identity.
On your Azure AI multi-service resource, assign the identity to the Cognitive Services User role.
Using the Azure portal, or the Skillset 2024-11-01-preview REST API, or an Azure SDK beta package that provides the syntax, configure a skillset to use an identity:
- The managed identity used on the connection belongs to the search service.
- The identity can be system managed or user assigned.
- The identity must have Cognitive Services User permissions on the Azure AI resource.
- @odata.type is always #Microsoft.Azure.Search.AIServicesByIdentity.
- subdomainUrl is the endpoint of your Azure AI multi-service resource. It can be in any region that's jointly supported by Azure AI Search and Azure AI services.

As with keys, the details you provide about the Azure AI Services resource are used for billing, not connections. All API requests made by Azure AI Search to Azure AI services for built-in skills processing continue to be internal and managed by Microsoft.

Example: system-assigned managed identity

Identity is set to null.

POST https://[service-name].search.windows.net/skillsets/[skillset-name]?api-version=2024-11-01-Preview  

{  
    "name": "my skillset name",  
    "skills":   
    [  
      // skills definition goes here 
    ],  
    "cognitiveServices": {  
        "@odata.type": "#Microsoft.Azure.Search.AIServicesByIdentity",  
        "description": "",  
        "subdomainUrl": “https://[subdomain-name].cognitiveservices.azure.com",  
        "identity": null 
    }  
}

Example: user-assigned managed identity

Identity is set to the resource ID of the user-assigned managed identity. To find an existing user-assigned managed identity, see Manage user-assigned managed identities.

For a user-assigned managed identity, set the @odata.type and the userAssignedIdentity properties.

POST https://[service-name].search.windows.net/skillsets/[skillset-name]?api-version=2024-11-01-Preview  

{  
    "name": "my skillset name",  
    "skills":   
    [  
      // skills definition goes here 
    ],  
    "cognitiveServices": {  
        "@odata.type": "#Microsoft.Azure.Search.AIServicesByIdentity",  
        "description": "",  
        "subdomainUrl": “https://[subdomain-name].cognitiveservices.azure.com",  
        "identity": {   
            "@odata.type":  "#Microsoft.Azure.Search.DataUserAssignedIdentity",   
            "userAssignedIdentity": ""/subscriptions/{subscription-ID}/resourceGroups/{resource-group-name}/providers/Microsoft.ManagedIdentity/userAssignedIdentities/{user-assigned-managed-identity-name}"" 
        }
    } 
}

Bill through a resource key

Azure AI Search can also charge for transaction using the Azure AI multi-service resource key. This approach is the default and is generally available. You can use the Azure portal, REST API, or an Azure SDK to add the key to a skillset.

You only need to add the key, not the subdomain or endpoint. If you leave the cognitiveServices property unspecified, your search service attempts to use the free enrichments available to your indexer on a daily basis. Execution of billable skills stops at 20 transactions per indexer invocation and a "Time Out" message appears in indexer execution history.

Sign in to the Azure portal.
Create an Azure AI multi-service resource in the same region as your search service.
Get the resource key from the Resources > Keys and endpoint page.
Add the key to a skillset definition:
- If using an Import data wizard, create or select the Azure AI account. The wizard adds the resource key to your skillset definition.
- For a new or existing skillset, provide the key in skillset definition.

Use the Create or Update Skillset API, specifying cognitiveServices section in the body of the request:

PUT https://[servicename].search.windows.net/skillsets/[skillset name]?api-version=2024-07-01
api-key: [admin key]
Content-Type: application/json
{
    "name": "skillset name",
    "skills": 
    [
      {
        "@odata.type": "#Microsoft.Skills.Text.V3.EntityRecognitionSkill",
        "categories": [ "Organization" ],
        "defaultLanguageCode": "en",
        "inputs": [
          {
            "name": "text", "source": "/document/content"
          }
        ],
        "outputs": [
          {
            "name": "organizations", "targetName": "organizations"
          }
        ]
      }
    ],
    "cognitiveServices": {
        "@odata.type": "#Microsoft.Azure.Search.CognitiveServicesByKey",
        "description": "mycogsvcs",
        "key": "<your key goes here>"
    }
}

The following code snippet is from azure-search-dotnet-samples, trimmed for brevity.

IConfigurationBuilder builder = new ConfigurationBuilder().AddJsonFile("appsettings.json");
IConfigurationRoot configuration = builder.Build();

string searchServiceUri = configuration["SearchServiceUri"];
string adminApiKey = configuration["SearchServiceAdminApiKey"];
string cognitiveServicesKey = configuration["CognitiveServicesKey"];

SearchIndexerClient indexerClient = new SearchIndexerClient(new Uri(searchServiceUri), new AzureKeyCredential(adminApiKey));

// Create the skills
Console.WriteLine("Creating the skills...");
OcrSkill ocrSkill = CreateOcrSkill();
MergeSkill mergeSkill = CreateMergeSkill();

// Create the skillset
Console.WriteLine("Creating or updating the skillset...");
List<SearchIndexerSkill> skills = new List<SearchIndexerSkill>();
skills.Add(ocrSkill);
skills.Add(mergeSkill);

SearchIndexerSkillset skillset = CreateOrUpdateDemoSkillSet(indexerClient, skills, cognitiveServicesKey);

Remove the key

Enrichments are billable operations. If you no longer need to call Azure AI services, follow these instructions to remove the multi-service key and prevent use of the external resource. Without the key, the skillset reverts to the default allocation of 20 free transactions per indexer, per day. Execution of billable skills stops at 20 transactions and a "Time Out" message appears in indexer execution history when the allocation is used up.

Azure portal
REST

Sign in to the Azure portal.
Under Search management > Skillsets, select a skillset from the list.
Scroll to the section in the file containing "cognitiveServices".
Delete the key value from the JSON and save the skillset.

Get Skillset so that you have the full definition.
Formulate an Update Skillset request, providing the JSON definition of the skillset.

Remove the key in the body of the definition, and then send the request:

PUT https://[servicename].search.windows.net/skillsets/[skillset name]?api-version=2024-07-01
api-key: [admin key]
Content-Type: application/json
{
    "name": "skillset name",
    "skills": 
    [
      {
        "@odata.type": "#Microsoft.Skills.Text.V3.EntityRecognitionSkill",
        "categories": [ "Organization" ],
        "defaultLanguageCode": "en",
        "inputs": [
          {
            "name": "text", "source": "/document/content"
          }
        ],
        "outputs": [
          {
            "name": "organizations", "targetName": "organizations"
          }
        ]
      }
    ],
    "cognitiveServices": {
        "@odata.type": "#Microsoft.Azure.Search.CognitiveServicesByKey",
        "description": "mycogsvcs",
        "key": ""
    }
}

Alternatively, you can set "cognitiveServices" to null:

"cognitiveServices": null,

How the key is used

Billing goes into effect when API calls to Azure AI services resources exceed 20 API calls per indexer, per day. You can reset the indexer to reset the API count.

Keyless and key-based connections are used for billing, but not for enrichment operations' connections. For connections, a search service connects over the internal network to an Azure AI services resource that's located in the same physical region. Most regions that offer Azure AI Search also offer other Azure AI services such as Language. If you attempt AI enrichment in a region that doesn't have both services, you'll see this message: "Provided key isn't a valid CognitiveServices type key for the region of your search service."

Indexers can be configured to run in a private execution environment for dedicated processing using just the search nodes of your own search service. Even if you're using private execution environment, Azure AI Search still uses its internally provisioned Azure AI multiservice resource to perform all skill enrichments.

Note

Some built-in skills are based on non-regional Azure AI services (for example, the Text Translation Skill). Using a non-regional skill means that your request might be serviced in a region other than the Azure AI Search region. For more information on non-regional services, see the Azure AI services product by region page.

Public connection requirements

Depending on when your search service was created, and its tier and region, billing for built-in skills can require a public connection from Azure AI Search to Azure AI multi-service. Disabling public network access breaks billing in some scenarios. Review the requirements for connections through a shared private link to determine whether your search service requires a public connection.

If you can't use the public network, you can configure a Custom Web API skill implemented with an Azure Function that supports private endpoints and add the Azure AI services resource to the same VNET. In this way, you can call Azure AI services resource directly from the custom skill using private endpoints.

Key requirements special cases

Custom Entity Lookup is metered by Azure AI Search, not Azure AI services, but it requires an Azure AI multi-service resource key to unlock transactions beyond 20 per indexer, per day. For this skill only, the resource key unblocks the number of transactions, but is unrelated to billing.

Free enrichments

AI enrichment offers a small quantity of free processing of billable enrichments so that you can complete short exercises without having to attach an Azure AI multi-service resource. Free enrichments are 20 documents per day, per indexer. You can reset the indexer to reset the counter if you want to repeat an exercise.

Some enrichments are always free:

Utility skills that don't call Azure AI services (namely, Conditional, Document Extraction, Shaper, Text Merge, and Text Split skills) aren't billable.
Text extraction from PDF documents and other application files is nonbillable. Text extraction, which occurs during the document cracking, isn't an AI enrichment, but it occurs during AI enrichment and is thus noted here.

Billable enrichments

During AI enrichment, Azure AI Search calls the Azure AI services APIs for built-in skills that are based on Azure AI Vision, Translator, and Azure AI Language.

Billable built-in skills that make backend calls to Azure AI services include Entity Linking, Entity Recognition, Image Analysis, Key Phrase Extraction, Language Detection, OCR, Personally Identifiable Information (PII) Detection, Sentiment, and Text Translation.

Image extraction is an Azure AI Search operation that occurs when documents are cracked prior to enrichment. Image extraction is billable on all tiers, except for 20 free daily extractions on the free tier. Image extraction costs apply to image files inside blobs, embedded images in other files (PDF and other app files), and for images extracted using Document Extraction. For image extraction pricing, see the Azure AI Search pricing page.

Tip

To lower the cost of skillset processing, enable incremental enrichment to cache and reuse any enrichments that are unaffected by changes made to a skillset. Caching requires Azure Storage (see pricing but the cumulative cost of skillset execution is lower if existing enrichments can be reused, especially for skillsets that use image extraction and analysis.

Example: Estimate costs

To estimate the costs associated with Azure AI Search indexing, start with an idea of what an average document looks like so you can run some numbers. For example, you might approximate:

1,000 PDFs.
Six pages each.
One image per page (6,000 images).
3,000 characters per page.

Assume a pipeline that consists of document cracking of each PDF, image and text extraction, optical character recognition (OCR) of images, and entity recognition of organizations.

The prices shown in this article are hypothetical. They're used to illustrate the estimation process. Your costs could be lower. For the actual price of transactions, see Azure AI services pricing.

For document cracking with text and image content, text extraction is currently free. For 6,000 images, assume $1 for every 1,000 images extracted. That's a cost of $6.00 for this step.
For OCR of 6,000 images in English, the OCR cognitive skill uses the best algorithm (DescribeText). Assuming a cost of $2.50 per 1,000 images to be analyzed, you would pay $15.00 for this step.
For entity extraction, you'd have a total of three text records per page. Each record is 1,000 characters. Three text records per page multiplied by 6,000 pages equal 18,000 text records. Assuming $2.00 per 1,000 text records, this step would cost $36.00.

Putting it all together, you'd pay about $57.00 to ingest 1,000 PDF documents of this type with the described skillset.

Share via

Attach an Azure AI multi-service resource to a skillset in Azure AI Search

Prerequisites

Bill through a keyless connection

Example: system-assigned managed identity

Example: user-assigned managed identity

Bill through a resource key

Remove the key

How the key is used

Public connection requirements

Key requirements special cases

Free enrichments

Billable enrichments

Example: Estimate costs

Next steps

Feedback

Additional resources