Edit

Share via

The Azure AI Foundry SDK

  • Article

The Azure AI Foundry SDK is a comprehensive toolchain designed to simplify the development of AI applications on Azure. It enables developers to:

  • Access popular models from various model providers through a single interface
  • Easily combine together models, data, and AI services to build AI-powered applications
  • Evaluate, debug, and improve application quality & safety across development, testing, and production environments

The AI Foundry SDK is a set of packages and services designed to work together. You can use the Azure AI Projects client library to easily use multiple services through a single project client and connection string. You can also use services and SDKs on their own and connect directly to your services.

If you want to jump right in and start building an app, check out:

Get started with Projects

The best way to get started using the Azure AI Foundry SDK is by using a project. AI projects connect together different data, assets, and services you need to build AI applications. The AI project client allows you to easily access these project components from your code by using a single connection string.

First follow steps to create an AI Project if you don't have one already.

Sign in with the Azure CLI using the same account that you use to access your AI Project:

az login

Install the Azure AI projects client library:

pip install azure-ai-projects azure-identity

Create a project client in code:

from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient

project_connection_string="your_connection_string"

project = AIProjectClient.from_connection_string(
  conn_str=project_connection_string,
  credential=DefaultAzureCredential())

dotnet add package Azure.AI.Projects
dotnet add package Azure.Identity

Add using statements:

using Azure.Identity;
using Azure.AI.Projects;

Create a project client in code:

var connectionString = "<your_connection_string>";
var projectClient = new AIProjectClient(connectionString, new DefaultAzureCredential());

Copy the Project connection string from the Overview page of the project and update the connections string value above.

Once you have created the project client, you can use the client for the capabilities in the following sections.

Be sure to check out the reference and samples.

Be sure to check out the reference and samples.

Azure OpenAI Service

The Azure OpenAI Service provides access to OpenAI's models including the GPT-4o, GPT-4o mini, GPT-4, GPT-4 Turbo with Vision, DALLE-3, Whisper, and Embeddings model series with the data residency, scalability, safety, security and enterprise capabilities of Azure.

If you have code that uses the OpenAI SDK, you can easily target your code to use the Azure OpenAI service. First, install the OpenAI SDK:

pip install openai

If you have existing code that uses the OpenAI SDK, you can use the project client to create an AzureOpenAI client that uses your project's Azure OpenAI connection:

openai = project.inference.get_azure_openai_client(api_version="2024-06-01")
response = openai.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "You are a helpful writing assistant"},
        {"role": "user", "content": "Write me a poem about flowers"},
    ]
)

print(response.choices[0].message.content)

dotnet add package Azure.AI.OpenAI

Add using statements:

using OpenAI.Chat;
using Azure.AI.OpenAI;

If you have existing code that uses the OpenAI SDK, you can use the project client to create an AzureOpenAI client that uses your project's Azure OpenAI connection:

var connections = projectClient.GetConnectionsClient();
ConnectionResponse connection = connections.GetDefaultConnection(ConnectionType.AzureOpenAI, withCredential: true);
var properties = connection.Properties as ConnectionPropertiesApiKeyAuth;

if (properties == null) {
    throw new Exception("Invalid auth type, expected API key auth");
}

// Create and use an Azure OpenAI client
AzureOpenAIClient azureOpenAIClient = new(
    new Uri(properties.Target),
    new AzureKeyCredential(properties.Credentials.Key));

// This must match the custom deployment name you chose for your model
ChatClient chatClient = azureOpenAIClient.GetChatClient("gpt-4o-mini");

ChatCompletion completion = chatClient.CompleteChat(
    [
        new SystemChatMessage("You are a helpful assistant that talks like a pirate."),
        new UserChatMessage("Does Azure OpenAI support customer managed keys?"),
        new AssistantChatMessage("Yes, customer managed keys are supported by Azure OpenAI"),
        new UserChatMessage("Do other Azure AI services support this too?")
    ]);

Console.WriteLine($"{completion.Role}: {completion.Content[0].Text}");

If you’re already using the Azure OpenAI SDK directly against the Azure OpenAI Service, the project provides a convenient way to use Azure OpenAI Service capabilities alongside the rest of the AI Foundry capabilities.

Azure AI model inference service

The Azure AI model inference service offers access to powerful models from leading providers like OpenAI, Microsoft, Meta, and more. These models support tasks such as content generation, summarization, and code generation.

To use the model inference service, first ensure that your project has an AI Services connection (in the management center).

Install the azure-ai-inference client library:

pip install azure-ai-inference

You can use the project client to get a configured and authenticated ChatCompletionsClient or EmbeddingsClient:

# get an chat inferencing client using the project's default model inferencing endpoint
chat = project.inference.get_chat_completions_client()

# run a chat completion using the inferencing client
response = chat.complete(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "You are a helpful writing assistant"},
        {"role": "user", "content": "Write me a poem about flowers"},
    ]
)

print(response.choices[0].message.content)

dotnet add package Azure.AI.Inference

Add using statements:

using Azure.AI.Inference;

You can use the project client to get a configured and authenticated ChatCompletionsClient or EmbeddingsClient:

var connectionString = Environment.GetEnvironmentVariable("AIPROJECT_CONNECTION_STRING");
var projectClient = new AIProjectClient(connectionString, new DefaultAzureCredential());

ChatCompletionsClient chatClient = projectClient.GetChatCompletionsClient();

var requestOptions = new ChatCompletionsOptions()
{
    Messages =
        {
            new ChatRequestSystemMessage("You are a helpful assistant."),
            new ChatRequestUserMessage("How many feet are in a mile?"),
        },
    Model = "gpt-4o-mini"
};

Response<ChatCompletions> response = chatClient.Complete(requestOptions);
Console.WriteLine(response.Value.Content);

You can change the model name to any model that you deployed to the inference service or Azure OpenAI service.

To learn more about using the Azure AI inferencing client, check out the Azure AI model inferencing reference.

Prompt Templates

The inferencing client supports for creating prompt messages from templates. The template allows you to dynamically generate prompts using inputs that are available at runtime.

To use prompt templates, install the azure-ai-inference package:

pip install azure-ai-inference

You can render a prompt template from an inline string:

from azure.ai.inference.prompts import PromptTemplate

# create a prompt template from an inline string (using mustache syntax)
prompt_template = PromptTemplate.from_string(prompt_template="""
    system:
    You are a helpful writing assistant.
    The user's first name is {{first_name}} and their last name is {{last_name}}.

    user:
    Write me a poem about flowers
    """)

# generate system message from the template, passing in the context as variables
messages = prompt_template.create_messages(first_name="Jane", last_name="Doe")
print(messages)

Note

Leading whitespace is automatically trimmed from input strings.

This code outputs messages that you can then pass to a chat completion call:

[
  {'role': 'system', 'content': "You are a helpful writing assistant.\nThe user's first name is Jane and their last name is Doe."}
  {'role': 'user', 'content': 'Write me a poem about flowers'}
]

You can also load prompts from a Prompty file, enabling you to also load the model name and parameters from the .prompty file:

from azure.ai.inference.prompts import PromptTemplate

prompt_template = PromptTemplate.from_prompty("myprompt.prompty")
messages = prompt_template.create_messages(first_name="Jane", last_name="Doe")

response = chat.complete(
    messages=messages,
    model=prompt_template.model_name,
    **prompt_template.parameters,
)

If you have an Azure AI Search resource connected to your project, you can also use the project client to create an Azure AI Search client using the project connection.

Install the Azure AI Search client library:

pip install azure-search-documents

Instantiate the search and/or search index client as desired:

from azure.core.credentials import AzureKeyCredential
from azure.ai.projects.models import ConnectionType
from azure.search.documents import SearchClient
from azure.search.documents.indexes import SearchIndexClient

# use the project client to get the default search connection
search_connection = project.connections.get_default(
    connection_type=ConnectionType.AZURE_AI_SEARCH,
    with_credentials=True)

# Create a client to create and manage search indexes
index_client = SearchIndexClient(
    endpoint=search_connection.endpoint_url,
    credential=AzureKeyCredential(key=search_connection.key)
)

# Create a client to run search queries
search_client = SearchClient(
    index_name="your_index_name",
    endpoint=search_connection.endpoint_url,
    credential=AzureKeyCredential(key=search_connection.key)
)

dotnet add package Azure.Search.Documents

Add using statements:

using Azure.Search.Documents;
using Azure.Search.Documents.Models;

Instantiate the search and/or search index client as desired:

var connections = projectClient.GetConnectionsClient();
var connection = connections.GetDefaultConnection(ConnectionType.AzureAISearch, withCredential: true).Value;

var properties = connection.Properties as ConnectionPropertiesApiKeyAuth;
if (properties == null) {
    throw new Exception("Invalid auth type, expected API key auth");
}

SearchClient searchClient = new SearchClient(
    new Uri(properties.Target),
    "products",
    new AzureKeyCredential(properties.Credentials.Key));

To learn more about using Azure AI Search, check out Azure AI Search documentation.

Azure AI Agent Service

Azure AI Agent Service is a fully managed service designed to empower developers to securely build, deploy, and scale high-quality, and extensible AI agents. Using an extensive ecosystem of models, tools and capabilities from OpenAI, Microsoft, and third-party providers, Azure AI Agent Service enables building agents for a wide range of generative AI use cases.

To get access to agents, sign-up for the preview.

Evaluation

You can use the project client to easily connect to the Azure AI evaluation service, and models needed for running your evaluators.

pip install azure-ai-evaluation

Using the project.scope parameter, we can instantiate a ViolenceEvaluator:

from azure.ai.evaluation import ViolenceEvaluator
from azure.identity import DefaultAzureCredential

# Initializing Violence Evaluator with project information
violence_eval = ViolenceEvaluator(
    azure_ai_project=project.scope,
    credential=DefaultAzureCredential())

# Running Violence Evaluator on single input row
violence_score = violence_eval(query="what's the capital of france", response="Paris")
print(violence_score)

NOTE: to run violence evaluators your project needs to be in East US 2, Sweden Central, US North Central, France Central.

To learn more, check out Evaluation using the SDK.

An Azure AI evaluation package is not yet available for C#. For a sample on how to use Prompty and Semantic Kernel for evaluation, see the contoso-chat-csharp-prompty sample.

Tracing

To enable tracing, first ensure your project has an attached Application Insights resource. Go to the Tracing page of your project and follow instructions to create or attach Application Insights.

Install the Azure Monitor OpenTelemetry package:

pip install azure-monitor-opentelemetry

Use the following code to enable instrumentation of the Azure AI Inference SDK and logging to your AI project:

# Enable instrumentation of AI packages (inference, agents, openai, langchain)
project.telemetry.enable()

# Log traces to the project's application insights resource
application_insights_connection_string = project.telemetry.get_connection_string()
if application_insights_connection_string:
    configure_azure_monitor(connection_string=application_insights_connection_string)

Tracing is not yet integrated into the projects package. For instructions on how to instrument and log traces from the Azure AI Inferencing package, see azure-sdk-for-dotnet.

Related content

Below are some helpful links to other services and frameworks that you can use with the Azure AI Foundry SDK.

Azure AI Services

Client libraries:

Management libraries:

Frameworks

Azure Machine Learning

Prompt flow

Semantic Kernel