Token consumption

Question

Hi, I wanted to check how many tokens are being used up by different models in azure openai, in the metrics section I am not able to see the exact token usage. On the count aggeration it is giving 4 what is that suppose to indicate.

I have option for active token and prompt completion token, but I want to see how many are being used. And why is active tokens max=0? is there a way by which I can see how every model is generating the token - individual data.

Answer 1

Hello Mansi Gusain,

Welcome to the Microsoft Q&A and thank you for posting your questions here.

I understand that you would like to know how many tokens are being used up by different models in Azure OpenAI.

To breakdown your questions:

1. Count Aggregation Showing 4: This typically means that there are four data points being aggregated. It doesn't directly indicate the number of tokens used but rather the number of records or events being counted.
2. Active Tokens Max=0: This could indicate that there are no active tokens being used at the time of measurement. Active tokens usually refer to tokens currently in use during an ongoing session or request.
3. Seeing Token Usage for Each Model: To get detailed token usage for each model, you can use the following steps:
   1. Metric Namespace: Select the appropriate namespace related to Azure OpenAI.
   2. Metric: Choose metrics like "Prompt Tokens" and "Completion Tokens" to see the tokens used for prompts and completions.
   3. Aggregation: Use aggregations like "Sum" to get the total number of tokens used over a period.

For more granular data, you might need to use Azure Monitor logs or Application Insights to track and analyze token usage per request. This can give you detailed insights into how each model is generating and using tokens.

I hope this is helpful! Do not hesitate to let me know if you have any other questions.

---

Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful.