Share via

Text Embedding generation in Semantic Kernel

  • Article

With text embedding generation, you can use an AI model to generate vectors (aka embeddings). These vectors encode the semantic meaning of the text in such a way that mathematical equations can be used on two vectors to compare the similiarty of the original text. This is useful for scenarios such as Retrieval Augmented Generation (RAG), where we want to search a database of information for text related to a user query. Any matching information can then be provided as input to Chat Completion, so that the AI Model has more context when answering the user query.

When choosing an embedding model, you will need to consider the following:

  • What is the size of the vectors generated by the model, and is it configurable, as this will affect your vector storage cost.
  • What type of elements does the generated vectors contain, e.g. float32, float16, etc, as this will affect your vector storage cost.
  • How fast does it generate vectors?
  • How much does generation cost?

Tip

For more information about storing and searching vectors see What are Semantic Kernel Vector Store connectors?

Tip

For more information about using RAG with vector stores in Semantic Kernel, see How to use Vector Stores with Semantic Kernel Text Search and What are Semantic Kernel Text Search plugins?

Setting up your local environment

Some of the AI Services can be hosted locally and may require some setup. Below are instructions for those that support this.

No local setup.

Installing the necessary packages

Before adding embedding generation to your kernel, you will need to install the necessary packages. Below are the packages you will need to install for each AI service provider.

dotnet add package Microsoft.SemanticKernel.Connectors.AzureOpenAI

Creating text embedding generation services

Now that you've installed the necessary packages, you can create a text embedding generation service. Below are the several ways you can text create embedding generation services using Semantic Kernel.

Adding directly to the kernel

To add a text embedding generation service, you can use the following code to add it to the kernel's inner service provider.

Important

The Azure OpenAI embedding generation connector is currently experimental. To use it, you will need to add #pragma warning disable SKEXP0010.

using Microsoft.SemanticKernel;

#pragma warning disable SKEXP0010
IKernelBuilder kernelBuilder = Kernel.CreateBuilder();
kernelBuilder.AddAzureOpenAITextEmbeddingGeneration(
    deploymentName: "NAME_OF_YOUR_DEPLOYMENT", // Name of deployment, e.g. "text-embedding-ada-002".
    endpoint: "YOUR_AZURE_ENDPOINT",           // Name of Azure Open AI service endpoint, e.g. https://myaiservice.openai.azure.com.
    apiKey: "YOUR_API_KEY",
    modelId: "MODEL_ID",          // Optional name of the underlying model if the deployment name doesn't match the model name, e.g. text-embedding-ada-002.
    serviceId: "YOUR_SERVICE_ID", // Optional; for targeting specific services within Semantic Kernel.
    httpClient: new HttpClient(), // Optional; if not provided, the HttpClient from the kernel will be used.
    dimensions: 1536              // Optional number of dimensions to generate embeddings with.
);
Kernel kernel = kernelBuilder.Build();

Using dependency injection

If you're using dependency injection, you'll likely want to add your text embedding generation services directly to the service provider. This is helpful if you want to create singletons of your embedding generation services and reuse them in transient kernels.

Important

The Azure OpenAI embedding generation connector is currently experimental. To use it, you will need to add #pragma warning disable SKEXP0010.

using Microsoft.SemanticKernel;

var builder = Host.CreateApplicationBuilder(args);

#pragma warning disable SKEXP0010
builder.Services.AddAzureOpenAITextEmbeddingGeneration(
    deploymentName: "NAME_OF_YOUR_DEPLOYMENT", // Name of deployment, e.g. "text-embedding-ada-002".
    endpoint: "YOUR_AZURE_ENDPOINT",           // Name of Azure Open AI service endpoint, e.g. https://myaiservice.openai.azure.com.
    apiKey: "YOUR_API_KEY",
    modelId: "MODEL_ID",          // Optional name of the underlying model if the deployment name doesn't match the model name, e.g. text-embedding-ada-002.
    serviceId: "YOUR_SERVICE_ID", // Optional; for targeting specific services within Semantic Kernel.
    dimensions: 1536              // Optional number of dimensions to generate embeddings with.
);

builder.Services.AddTransient((serviceProvider)=> {
    return new Kernel(serviceProvider);
});

Creating standalone instances

Lastly, you can create instances of the service directly so that you can either add them to a kernel later or use them directly in your code without ever injecting them into the kernel or in a service provider.

Important

The Azure OpenAI embedding generation connector is currently experimental. To use it, you will need to add #pragma warning disable SKEXP0010.

using Microsoft.SemanticKernel.Connectors.AzureOpenAI;

#pragma warning disable SKEXP0010
AzureOpenAITextEmbeddingGenerationService textEmbeddingGenerationService = new (
    deploymentName: "NAME_OF_YOUR_DEPLOYMENT", // Name of deployment, e.g. "text-embedding-ada-002".
    endpoint: "YOUR_AZURE_ENDPOINT",           // Name of Azure Open AI service endpoint, e.g. https://myaiservice.openai.azure.com.
    apiKey: "YOUR_API_KEY",
    modelId: "MODEL_ID",          // Optional name of the underlying model if the deployment name doesn't match the model name, e.g. text-embedding-ada-002.
    httpClient: new HttpClient(), // Optional; if not provided, the HttpClient from the kernel will be used.
    dimensions: 1536              // Optional number of dimensions to generate embeddings with.
);

Using text embedding generation services

All text embedding generation services implement the ITextEmbeddingGenerationService which has a single method GenerateEmbeddingsAsync that can generate ReadOnlyMemory<float> vectors from provided string values. An extension method GenerateEmbeddingAsync is also available for single value versions of the same action.

Here is an example of how to invoke the service with multiple values.

IList<ReadOnlyMemory<float>> embeddings =
    await textEmbeddingGenerationService.GenerateEmbeddingsAsync(
    [
        "sample text 1",
        "sample text 2"
    ]);

Here is an example of how to invoke the service with a single value.

using Microsoft.SemanticKernel.Embeddings;

ReadOnlyMemory<float> embedding =
    await textEmbeddingGenerationService.GenerateEmbeddingAsync("sample text");

Coming soon

More information coming soon.

Coming soon

More information coming soon.