Azure OpenAI – Agentic AI – Semantic Kernel – RAG integration

Question

I designed an ASP.NET Core WebAPI application which provides chatbot capabilities by searching for specific products through some internal documents.

I used for this application the RAG approach.

It’s working just fine.

Now I want to leverage the Azure Agentic AI capabilities by updating of this application to use the Azure Agentic AI  – Semantic Kernel approach.

Which means to have Azure Agentic AI on my own data with Semantic Kernel.

I installed the Azure AI SemanticKernel:

dotnet add package Microsoft.SemanticKernel

I created 2 plugin functions: CoveragePlugin and ProductCodePlugin to be called by the Semantic Kernel when some specific context found in searched documents.

I searched on internet/documentation/ ChatGPT but I could not find how to make the application bootstrap setup in such way that Azure OpenAI client (together with the underlying Azure Search Service) to be under the Semantic Kernel.

A version of the Program.cs section is like bellow (but this is not what I want since SearchIndexClient is not under the Semantic Kernel):

using Azure.Search.Documents.Indexes;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;

var builder = WebApplication.CreateBuilder(args);

builder.Services
    .AddSingleton((_) => new SearchIndexClient(
        new Uri("AZURE_SEARCH_SERVICE_ENDPOINT"),
        new Azure.AzureKeyCredential("AZURE_SEARCH_SERVICE_API_KEY")))
    .AddKernel()
    .AddAzureOpenAIChatCompletion(
        deploymentName: builder.Configuration.GetValue<string>("AZURE_OPENAI_DEPLOYMENT_NAME"),
        endpoint: builder.Configuration.GetValue<string>("AZURE_OPENAI_ENDPOINT"),
        apiKey: builder.Configuration.GetValue<string>("AZURE_OPENAI_API_KEY"))
    .Plugins
    .AddFromType<CoveragePlugin>("CoveragePlugin")
    .AddFromType<ProductCodePlugin>("ProductCodePlugin");

A version which potentially could work is mentioned on this blog where both Azure OpenAI and underlying Azure Search Service are included via OpenAIPromptExecutionSettings :   

var azureSearchExtensionConfiguration = new AzureSearchChatExtensionConfiguration
{
    SearchEndpoint = new Uri(Environment.GetEnvironmentVariable("AZURE_AI_SEARCH_ENDPOINT")),
    Authentication = new OnYourDataApiKeyAuthenticationOptions(Environment.GetEnvironmentVariable("AZURE_AI_SEARCH_API_KEY")),
    IndexName = Environment.GetEnvironmentVariable("AZURE_AI_SEARCH_INDEX")
};

var chatExtensionsOptions = new AzureChatExtensionsOptions { Extensions = { azureSearchExtensionConfiguration } };
var executionSettings = new OpenAIPromptExecutionSettings { AzureChatExtensionsOptions = chatExtensionsOptions };

But AzureSearchChatExtensionConfiguration is not recognized, even if here is clearly mentioned that AzureCognitiveSearchChatExtensionConfiguration is now AzureSearchChatExtensionConfiguration.

In the end I would like to search through our internal documents like this:

using ftos_chatbot_mvc.Models;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
namespace ftos_chatbot_mvc.Services
{
    public class ChatAgentService
    {
        private readonly ILogger<ChatService> _logger;
        private readonly IConfigurationService _config;
        private readonly IChatCompletionService _chatCompletionService;
        public ChatAgentService(ILogger<ChatService> logger,
        IConfigurationService config,
        IChatCompletionService chatCompletionService)
        {
            _logger = logger;
            _config = config;
            _chatCompletionService = chatCompletionService;
        }
        public async Task<ChatMessageContent> SendUserTextAsync(ChatRequest request)
        {
            return await _chatCompletionService.GetChatMessageContentAsync(request.Message);
        }
    }
}

Could you please guide me to include both Azure OpenAI and underlying Azure Search Service under the Azure Semantic Kernel?

Answer 1

Hello Octavian Mocanu,

Welcome to the Microsoft Q&A and thank you for posting your questions here.

I understand that you are having challenges to integrate both Azure OpenAI and Azure Search Service under the Azure Semantic Kernel.

There are two methods to achieve this, by leveraging necessary components (Microsoft.SemanticKernel and Microsoft.SemanticKernel.Connectors.AzureAISearch). Secondly, using the KernelBuilder for a more modular setup.

The below narrow down based on your scenario:

Get NuGet packages installed:

dotnet add package Microsoft.SemanticKernel
dotnet add package Microsoft.SemanticKernel.Connectors.AzureAISearch --prerelease

Implementation in Program.cs

using Azure;
using Azure.Search.Documents;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Connectors.AzureAISearch;
using Microsoft.SemanticKernel.ChatCompletion;
using Microsoft.SemanticKernel.Plugins;
var builder = WebApplication.CreateBuilder(args);
// Configure Semantic Kernel
builder.Services.AddKernel(kernel =>
{
    // Add Azure OpenAI Chat Completion
    kernel.Config.AddAzureChatCompletionService(
        deploymentName: builder.Configuration["AZURE_OPENAI_DEPLOYMENT_NAME"],
        endpoint: builder.Configuration["AZURE_OPENAI_ENDPOINT"],
        apiKey: builder.Configuration["AZURE_OPENAI_API_KEY"]);
    // Add Azure Search Vector Store
    kernel.Config.AddAzureAISearchVectorStore(
        searchEndpoint: new Uri(builder.Configuration["AZURE_SEARCH_SERVICE_ENDPOINT"]),
        credential: new AzureKeyCredential(builder.Configuration["AZURE_SEARCH_SERVICE_API_KEY"]));
    // Add Plugins
    kernel.Config.AddPlugin(new CoveragePlugin(), "CoveragePlugin");
    kernel.Config.AddPlugin(new ProductCodePlugin(), "ProductCodePlugin");
});
var app = builder.Build();
app.Run();

Each plugin should implement logic that interacts with Semantic Kernel's context. For example:

public class CoveragePlugin : IPlugin
{
    public string GetCoverageDetails(string query)
    {
        // Example implementation for retrieving coverage details
        return $"Coverage details for {query}.";
    }
}

public class ProductCodePlugin : IPlugin
{
    public string GetProductCode(string productName)
    {
        // Example implementation for retrieving product codes
        return $"Product code for {productName} is ABC123.";
    }
}

In the ChatAgentService, utilize the Semantic Kernel to process queries:

public class ChatAgentService
{
    private readonly IKernel _kernel;
    public ChatAgentService(IKernel kernel)
    {
        _kernel = kernel;
    }
    public async Task<string> QueryAsync(string userQuery)
    {
        var context = _kernel.CreateNewContext();
        context["userQuery"] = userQuery;
        var completion = await _kernel.RunAsync(context, "CoveragePlugin.GetCoverageDetails", "ProductCodePlugin.GetProductCode");
        return completion.Result;
    }
}

Ensure Azure Search results are pre-processed and embedded into the Semantic Kernel context:

public async Task<List<SearchResult>> SearchDocumentsAsync(string query)
{
    var searchClient = new SearchClient(new Uri(builder.Configuration["AZURE_SEARCH_SERVICE_ENDPOINT"]),
                                        builder.Configuration["AZURE_SEARCH_INDEX_NAME"],
                                        new AzureKeyCredential(builder.Configuration["AZURE_SEARCH_SERVICE_API_KEY"]));
    var results = await searchClient.SearchAsync<SearchResult>(query);
    return results.Value.ToList();
}

I hope this is helpful! Do not hesitate to let me know if you have any other questions.

---

Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful.