Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
3,732 questions
Tokens not getting cached in azure OpenAI.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(169.60000000000002, 66%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESR%3C/text%3E%3C/svg%3E)
Sharad Roy
0
Reputation points
When using gpt-4o model in azure OpenAI, cached tokesn count in usage is always coming as zero. The same prompt and history used directly in the openai API gives cached tokens correctly. Do we need to enable any settings to enable caching? I am suing the Openai nodejs sdk for both azure and direct openai.
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(281.6, 88%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESA%3C/text%3E%3C/svg%3E)
Saideep Anchuri 3,140 Reputation points Microsoft Vendor2025-02-27T09:43:12.4033333+00:00 Hi Sharad Roy
It seems you're having trouble with prompt caching in Azure OpenAI Service.
Here are some steps:
- Verify that the model you're utilizing is compatible with prompt caching. In Azure OpenAI, prompt caching is only supported by specific models such as gpt-4o-2024-11-20 and gpt-4o-mini-2024-07-18.
- The prompt must be a minimum of 1,024 tokens in length, and the first 1,024 tokens must be identical for caching to work.
- Prompt caching is enabled by default for supported models,
Kindly refer below link: prompt-caching
Thank You.
-
Saideep Anchuri 3,140 Reputation points Microsoft Vendor
2025-02-28T04:17:44.25+00:00
Sign in to comment
Sign in to answer