Hi, What is the ETA for the addition of the OpenAI Usage API to the OpenAI Services on Azure? I have several OpenAI instances but can't manage costs beyond instance/model costs. Considering enterprise scenarios, it's highly important for effective cost management by project/application. References: https://platform.openai.com/docs/api-reference/usage https://platform.openai.com/docs/api-reference/usage/costs

Hi @Andres da Silva Santos Thanks for posting your question in Microsoft Q&A. As of now, Azure OpenAI Service does not provide a dedicated Usage API like OpenAI’s platform. However, you can manage costs using Azure Cost Management and Billing, Provisioned Throughput Units (PTUs) for predictable pricing, and Azure Monitor for tracking usage and setting alerts. For more detailed information, please refer to the official Azure OpenAI Service documentation: Manage costs for Azure OpenAI Azure OpenAI Pricing If the response helped, please do click Accept Answer and Yes for was this answer helpful.

@Andres da Silva Santos The ETA for the addition of the OpenAI Usage API to the OpenAI Services on Azure is currently set for October 2024 . This API will provide detailed usage metrics, which will be crucial for effective cost management by project or application See: https://learn.microsoft.com/en-us/azure/ai-services/openai/monitor-openai-reference https://learn.microsoft.com/en-us/azure/ai-services/openai/

Azure OpenAI Usage API? - Microsoft Q&A

Accepted answer

Vikram Singh 655 Reputation points Microsoft Employee

2025-01-31T10:55:44.39+00:00
Hi @Andres da Silva Santos

Thanks for posting your question in Microsoft Q&A.

As of now, Azure OpenAI Service does not provide a dedicated Usage API like OpenAI’s platform. However, you can manage costs using Azure Cost Management and Billing, Provisioned Throughput Units (PTUs) for predictable pricing, and Azure Monitor for tracking usage and setting alerts.

For more detailed information, please refer to the official Azure OpenAI Service documentation:

Manage costs for Azure OpenAI

Azure OpenAI Pricing

If the response helped, please do click Accept Answer and Yes for was this answer helpful.
Please sign in to rate this answer.
Andres da Silva Santos 115 Reputation points

2025-01-31T12:41:35.98+00:00

Hello Vikram, thank you for your answer.

However, I still have doubts about how to monitor the cost at the application level, not at the resource level.

I know that the Azure resource has the usage cost of all deployed models individually, but how can I determine within a deployed model what percentage of consumption is from one client or another?

For example, OpenAI allows me to send project_id and provides the endpoint https://api.openai.com/v1/organization/costs which returns the following JSON with the project_id information:

{ "object": "page", "data": [ { "object": "bucket", "start_time": 1730419200, "end_time": 1730505600, "results": [ { "object": "organization.costs.result", "amount": { "value": 0.06, "currency": "usd" }, "line_item": null, "project_id": null } ] } ], "has_more": false, "next_page": null }

How can I perform this monitoring using Azure OpenAI Services?

Vikram Singh 655 Reputation points Microsoft Employee

2025-01-31T13:17:10.1066667+00:00

Hi @Andres da Silva Santos ,

Thanks for getting back.

Azure OpenAI does not directly support sending a project_id to determine the consumption percentage from different clients within a deployed model. However, you can achieve this by using Azure API Management. By setting up API Management in front of your Azure OpenAI service, you can track and manage token usage across multiple applications or clients. You can configure policies in Azure API Management to set token limits and emit token metrics based on various keys, such as subscription key or originating IP address. This setup helps you monitor and control the consumption of your Azure OpenAI service effectively.

For more details, you can refer:

Azure OpenAI Service REST API reference

Import an Azure OpenAI API

Overview of generative AI gateway capabilities in Azure API Management

If the reply was helpful please don't forget to upvote and/or accept as answer.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Azure OpenAI Usage API?

1 additional answer

Your answer