Structured outputs

Structured outputs make a model follow a JSON Schema definition that you provide as part of your inference API call. This is in contrast to the older JSON mode feature, which guaranteed valid JSON would be generated, but was unable to ensure strict adherence to the supplied schema. Structured outputs are recommended for function calling, extracting structured data, and building complex multi-step workflows.

Note

Currently structured outputs are not supported with:

Supported models

  • o3-mini version 2025-01-31
  • o1 version: 2024-12-17
  • gpt-4o-mini version: 2024-07-18
  • gpt-4o version: 2024-08-06

API support

Support for structured outputs was first added in API version 2024-08-01-preview. It is available in the latest preview APIs as well as the latest GA API: 2024-10-21.

Getting started

You can use Pydantic to define object schemas in Python. Depending on what version of the OpenAI and Pydantic libraries you're running you might need to upgrade to a newer version. These examples were tested against openai 1.42.0 and pydantic 2.8.2.

pip install openai pydantic --upgrade

If you are new to using Microsoft Entra ID for authentication see How to configure Azure OpenAI Service with Microsoft Entra ID authentication.

from pydantic import BaseModel
from openai import AzureOpenAI
from azure.identity import DefaultAzureCredential, get_bearer_token_provider

token_provider = get_bearer_token_provider(
    DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default"
)

client = AzureOpenAI(
  azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT"), 
  azure_ad_token_provider=token_provider,
  api_version="2024-10-21"
)


class CalendarEvent(BaseModel):
    name: str
    date: str
    participants: list[str]

completion = client.beta.chat.completions.parse(
    model="MODEL_DEPLOYMENT_NAME", # replace with the model deployment name of your gpt-4o 2024-08-06 deployment
    messages=[
        {"role": "system", "content": "Extract the event information."},
        {"role": "user", "content": "Alice and Bob are going to a science fair on Friday."},
    ],
    response_format=CalendarEvent,
)

event = completion.choices[0].message.parsed

print(event)
print(completion.model_dump_json(indent=2))

Output

name='Science Fair' date='Friday' participants=['Alice', 'Bob']
{
  "id": "chatcmpl-A1EUP2fAmL4SeB1lVMinwM7I2vcqG",
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "logprobs": null,
      "message": {
        "content": "{\n  \"name\": \"Science Fair\",\n  \"date\": \"Friday\",\n  \"participants\": [\"Alice\", \"Bob\"]\n}",
        "refusal": null,
        "role": "assistant",
        "function_call": null,
        "tool_calls": [],
        "parsed": {
          "name": "Science Fair",
          "date": "Friday",
          "participants": [
            "Alice",
            "Bob"
          ]
        }
      }
    }
  ],
  "created": 1724857389,
  "model": "gpt-4o-2024-08-06",
  "object": "chat.completion",
  "service_tier": null,
  "system_fingerprint": "fp_1c2eaec9fe",
  "usage": {
    "completion_tokens": 27,
    "prompt_tokens": 32,
    "total_tokens": 59
  }
}

Function calling with structured outputs

Structured Outputs for function calling can be enabled with a single parameter, by supplying strict: true.

Note

Structured outputs are not supported with parallel function calls. When using structured outputs set parallel_tool_calls to false.

from enum import Enum
from typing import Union
from pydantic import BaseModel
import openai
from openai import AzureOpenAI

client = AzureOpenAI(
  azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT"), 
  api_key=os.getenv("AZURE_OPENAI_API_KEY"),  
  api_version="2024-10-21"
)


class GetDeliveryDate(BaseModel):
    order_id: str

tools = [openai.pydantic_function_tool(GetDeliveryDate)]

messages = []
messages.append({"role": "system", "content": "You are a helpful customer support assistant. Use the supplied tools to assist the user."})
messages.append({"role": "user", "content": "Hi, can you tell me the delivery date for my order #12345?"}) 

response = client.chat.completions.create(
    model="MODEL_DEPLOYMENT_NAME", # replace with the model deployment name of your gpt-4o 2024-08-06 deployment
    messages=messages,
    tools=tools
)

print(response.choices[0].message.tool_calls[0].function)
print(response.model_dump_json(indent=2))

Getting started

Add the following packages to your project to work with Azure OpenAI:

  • Azure.AI.OpenAI: Provides an Azure OpenAI client with Azure specific functionality that builds on top of the standard OpenAI library dependency.
  • Azure.Identity: Provides Microsoft Entra ID token authentication support across the Azure SDK libraries.
  • Newtonsoft.Json.Schema: Provides helpful utilities for working with JSON schemas.
dotnet add package Azure.AI.OpenAI --prerelease
dotnet add package Azure.Identity
dotnet add package Newtonsoft.Json.Schema

If you are new to using Microsoft Entra ID for authentication see How to configure Azure OpenAI Service with Microsoft Entra ID authentication.

using Azure.AI.OpenAI;
using Azure.Identity;
using Newtonsoft.Json.Schema.Generation;
using OpenAI.Chat;
using System.ClientModel;

// Create the clients
string endpoint = GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT");

AzureOpenAIClient openAIClient = new(
    new Uri(endpoint),
    new DefaultAzureCredential());

var client = openAIClient.GetChatClient("gpt-4o");

// Create a chat with initial prompts
var chat = new List<ChatMessage>()
    {
         new SystemChatMessage("Extract the event information and projected weather."),
         new UserChatMessage("Alice and Bob are going to a science fair in Seattle on June 1st, 2025.")
    };

// Get the schema of the class for the structured response
JSchemaGenerator generator = new JSchemaGenerator();
var jsonSchema = generator.Generate(typeof(CalendarEvent)).ToString();

// Get a completion with structured output
var chatUpdates = client.CompleteChatStreamingAsync(
        chat,
        new ChatCompletionOptions()
        {
            ResponseFormat = ChatResponseFormat.CreateJsonSchemaFormat(
                        "calenderEvent",
                        BinaryData.FromString(jsonSchema))
        });

// Write the structured response
await foreach (var chatUpdate in chatUpdates)
{
    foreach (var contentPart in chatUpdate.ContentUpdate)
    {
        Console.Write(contentPart.Text);
    }
}

// The class for the structured response
public class CalendarEvent()
{
    public string Name { get; set; }
    public string Date { get; set; }
    public List<string> Participants { get; set; }
}

Function calling with structured outputs

Structured Outputs for function calling can be enabled with a single parameter, by supplying strict: true.

using Azure.AI.OpenAI;
using Newtonsoft.Json.Schema.Generation;
using OpenAI.Chat;
using System.ClientModel;

// Create the clients
string endpoint = GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT");

AzureOpenAIClient openAIClient = new(
    new Uri(endpoint),
    new DefaultAzureCredential());

var chatClient = openAIClient.GetChatClient("gpt-4o");

// Local function to be used by the assistant tooling
string GetTemperature(string location, string date)
{
    // Placeholder for Weather API
    if(location == "Seattle" && date == "2025-06-01")
    {
        return "75";
    }

    return "50";
}

// Create a tool to get the temperature
ChatTool GetTemperatureTool = ChatTool.CreateFunctionTool(
    functionName: nameof(GetTemperature),
    functionSchemaIsStrict: true,
    functionDescription: "Get the projected temperature by date and location.",
    functionParameters: BinaryData.FromBytes("""
        {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The location of the weather."
                },
                "date": {
                    "type": "string",
                    "description": "The date of the projected weather."
                }
            },
            "required": ["location", "date"],
            "additionalProperties": false  
        }
        """u8.ToArray())
);

// Create a chat with prompts
var chat = new List<ChatMessage>()
    {
         new SystemChatMessage("Extract the event information and projected weather."),
         new UserChatMessage("Alice and Bob are going to a science fair in Seattle on June 1st, 2025.")
    };

// Create a JSON schema for the CalendarEvent structured response
JSchemaGenerator generator = new JSchemaGenerator();
var jsonSchema = generator.Generate(typeof(CalendarEvent)).ToString();

// Get a chat completion from the AI model
var completion = chatClient.CompleteChat(
        chat,
        new ChatCompletionOptions()
        {
            ResponseFormat = ChatResponseFormat.CreateJsonSchemaFormat(
                "calenderEvent",
                BinaryData.FromString(jsonSchema)),
            Tools = { GetTemperatureTool }
        });

Console.WriteLine(completion.Value.ToolCalls[0].FunctionName);

// Structured response class
public class CalendarEvent()
{
    public string Name { get; set; }
    public string Date { get; set; }
    public string Temperature { get; set; }
    public List<string> Participants { get; set; }
}

Getting started

response_format is set to json_schema with strict: true set.

curl -X POST  https://YOUR_RESOURCE_NAME.openai.azure.com/openai/deployments/YOUR_MODEL_DEPLOYMENT_NAME/chat/completions?api-version=2024-10-21 \
  -H "api-key: $AZURE_OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
    -d '{
        "messages": [
                {"role": "system", "content": "Extract the event information."},
                {"role": "user", "content": "Alice and Bob are going to a science fair on Friday."}
            ],
            "response_format": {
                "type": "json_schema",
                "json_schema": {
                    "name": "CalendarEventResponse",
                    "strict": true,
                    "schema": {
                        "type": "object",
                        "properties": {
                            "name": {
                              "type": "string"
                            },
                            "date": {
                                "type": "string"
                            },
                            "participants": {
                                "type": "array",
                                "items": {
                                    "type": "string"
                                }
                            }
                        },
                        "required": [
                            "name",
                            "date",
                            "participants"
                        ],
                        "additionalProperties": false
                    }
                }
          }
  }'

Output:

{
  "id": "chatcmpl-A1HKsHAe2hH9MEooYslRn9UmEwsag",
  "object": "chat.completion",
  "created": 1724868330,
  "model": "gpt-4o-2024-08-06",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "{\n  \"name\": \"Science Fair\",\n  \"date\": \"Friday\",\n  \"participants\": [\"Alice\", \"Bob\"]\n}"
      },
      "logprobs": null,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 33,
    "completion_tokens": 27,
    "total_tokens": 60
  },
  "system_fingerprint": "fp_1c2eaec9fe"
}

Function calling with structured outputs

curl -X POST  https://YOUR_RESOURCE_NAME.openai.azure.com/openai/deployments/YOUR_MODEL_DEPLOYMENT_NAME/chat/completions?api-version=2024-10-21 \
  -H "api-key: $AZURE_OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant. The current date is August 6, 2024. You help users query for the data they are looking for by calling the query function."
    },
    {
      "role": "user",
      "content": "look up all my orders in may of last year that were fulfilled but not delivered on time"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "query",
        "description": "Execute a query.",
        "strict": true,
        "parameters": {
          "type": "object",
          "properties": {
            "table_name": {
              "type": "string",
              "enum": ["orders"]
            },
            "columns": {
              "type": "array",
              "items": {
                "type": "string",
                "enum": [
                  "id",
                  "status",
                  "expected_delivery_date",
                  "delivered_at",
                  "shipped_at",
                  "ordered_at",
                  "canceled_at"
                ]
              }
            },
            "conditions": {
              "type": "array",
              "items": {
                "type": "object",
                "properties": {
                  "column": {
                    "type": "string"
                  },
                  "operator": {
                    "type": "string",
                    "enum": ["=", ">", "<", ">=", "<=", "!="]
                  },
                  "value": {
                    "anyOf": [
                      {
                        "type": "string"
                      },
                      {
                        "type": "number"
                      },
                      {
                        "type": "object",
                        "properties": {
                          "column_name": {
                            "type": "string"
                          }
                        },
                        "required": ["column_name"],
                        "additionalProperties": false
                      }
                    ]
                  }
                },
                "required": ["column", "operator", "value"],
                "additionalProperties": false
              }
            },
            "order_by": {
              "type": "string",
              "enum": ["asc", "desc"]
            }
          },
          "required": ["table_name", "columns", "conditions", "order_by"],
          "additionalProperties": false
        }
      }
    }
  ]
}'

Supported schemas and limitations

Azure OpenAI structured outputs support the same subset of the JSON Schema as OpenAI.

Supported types

  • String
  • Number
  • Boolean
  • Integer
  • Object
  • Array
  • Enum
  • anyOf

Note

Root objects cannot be the anyOf type.

All fields must be required

All fields or function parameters must be included as required. In the example below location, and unit are both specified under "required": ["location", "unit"].

{
    "name": "get_weather",
    "description": "Fetches the weather in the given location",
    "strict": true,
    "parameters": {
        "type": "object",
        "properties": {
            "location": {
                "type": "string",
                "description": "The location to get the weather for"
            },
            "unit": {
                "type": "string",
                "description": "The unit to return the temperature in",
                "enum": ["F", "C"]
            }
        },
        "additionalProperties": false,
        "required": ["location", "unit"]
    }

If needed, it's possible to emulate an optional parameter by using a union type with null. In this example, this is achieved with the line "type": ["string", "null"],.

{
    "name": "get_weather",
    "description": "Fetches the weather in the given location",
    "strict": true,
    "parameters": {
        "type": "object",
        "properties": {
            "location": {
                "type": "string",
                "description": "The location to get the weather for"
            },
            "unit": {
                "type": ["string", "null"],
                "description": "The unit to return the temperature in",
                "enum": ["F", "C"]
            }
        },
        "additionalProperties": false,
        "required": [
            "location", "unit"
        ]
    }
}

Nesting depth

A schema may have up to 100 object properties total, with up to five levels of nesting

additionalProperties: false must always be set in objects

This property controls if an object can have additional key value pairs that weren't defined in the JSON Schema. In order to use structured outputs, you must set this value to false.

Key ordering

Structured outputs are ordered the same as the provided schema. To change the output order, modify the order of the schema that you send as part of your inference request.

Unsupported type-specific keywords

Type Unsupported Keyword
String minlength
maxLength
pattern
format
Number minimum
maximum
multipleOf
Objects patternProperties
unevaluatedProperties
propertyNames
minProperties
maxProperties
Arrays unevaluatedItems
contains
minContains
maxContains
minItems
maxItems
uniqueItems

Nested schemas using anyOf must adhere to the overall JSON Schema subset

Example supported anyOf schema:

{
    "type": "object",
    "properties": {
        "item": {
            "anyOf": [
                {
                    "type": "object",
                    "description": "The user object to insert into the database",
                    "properties": {
                        "name": {
                            "type": "string",
                            "description": "The name of the user"
                        },
                        "age": {
                            "type": "number",
                            "description": "The age of the user"
                        }
                    },
                    "additionalProperties": false,
                    "required": [
                        "name",
                        "age"
                    ]
                },
                {
                    "type": "object",
                    "description": "The address object to insert into the database",
                    "properties": {
                        "number": {
                            "type": "string",
                            "description": "The number of the address. Eg. for 123 main st, this would be 123"
                        },
                        "street": {
                            "type": "string",
                            "description": "The street name. Eg. for 123 main st, this would be main st"
                        },
                        "city": {
                            "type": "string",
                            "description": "The city of the address"
                        }
                    },
                    "additionalProperties": false,
                    "required": [
                        "number",
                        "street",
                        "city"
                    ]
                }
            ]
        }
    },
    "additionalProperties": false,
    "required": [
        "item"
    ]
}

Definitions are supported

Supported example:

{
    "type": "object",
    "properties": {
        "steps": {
            "type": "array",
            "items": {
                "$ref": "#/$defs/step"
            }
        },
        "final_answer": {
            "type": "string"
        }
    },
    "$defs": {
        "step": {
            "type": "object",
            "properties": {
                "explanation": {
                    "type": "string"
                },
                "output": {
                    "type": "string"
                }
            },
            "required": [
                "explanation",
                "output"
            ],
            "additionalProperties": false
        }
    },
    "required": [
        "steps",
        "final_answer"
    ],
    "additionalProperties": false
}

Recursive schemas are supported

Example using # for root recursion:

{
        "name": "ui",
        "description": "Dynamically generated UI",
        "strict": true,
        "schema": {
            "type": "object",
            "properties": {
                "type": {
                    "type": "string",
                    "description": "The type of the UI component",
                    "enum": ["div", "button", "header", "section", "field", "form"]
                },
                "label": {
                    "type": "string",
                    "description": "The label of the UI component, used for buttons or form fields"
                },
                "children": {
                    "type": "array",
                    "description": "Nested UI components",
                    "items": {
                        "$ref": "#"
                    }
                },
                "attributes": {
                    "type": "array",
                    "description": "Arbitrary attributes for the UI component, suitable for any element",
                    "items": {
                        "type": "object",
                        "properties": {
                            "name": {
                                "type": "string",
                                "description": "The name of the attribute, for example onClick or className"
                            },
                            "value": {
                                "type": "string",
                                "description": "The value of the attribute"
                            }
                        },
                      "additionalProperties": false,
                      "required": ["name", "value"]
                    }
                }
            },
            "required": ["type", "label", "children", "attributes"],
            "additionalProperties": false
        }
    }

Example of explicit recursion:

{
    "type": "object",
    "properties": {
        "linked_list": {
            "$ref": "#/$defs/linked_list_node"
        }
    },
    "$defs": {
        "linked_list_node": {
            "type": "object",
            "properties": {
                "value": {
                    "type": "number"
                },
                "next": {
                    "anyOf": [
                        {
                            "$ref": "#/$defs/linked_list_node"
                        },
                        {
                            "type": "null"
                        }
                    ]
                }
            },
            "additionalProperties": false,
            "required": [
                "next",
                "value"
            ]
        }
    },
    "additionalProperties": false,
    "required": [
        "linked_list"
    ]
}