Can I automatically deploy and de-deploy a fine tuned openai model to reduce costs?

Question

I wish to use a fine tuned openai model. But it will not be used very frequently. Therefore the hourly hosting cost is a deal breaker and I really wish MS would mirror the OpenAI approach where the tokens are just billed at a higher rate.... But Is it possible to have a script that deploys the model when needed and afterwards deletes it again, so it does not incur costs before it is needed again?

Answer 1

Hi @83533917,

Thank you for reaching out to Microsoft Q&A forum!

It is not currently feasible to automatically deploy and de-deploy a fine-tuned Azure OpenAI model in real time to reduce hosting costs, as deployments require resource provisioning that takes several minutes. When fine-tuned model is deployed, it continues to incur an hourly cost regardless of whether you're actively using it. While scripts can automate deployment and deletion, the delay in availability makes on-demand usage impractical.

To manage costs effectively, use the Azure Pricing Calculator for estimation, monitor spending trends with Azure Cost Management tools, and set budgets and alerts for better cost control.

For more info: Plan to manage costs for Azure OpenAI Service.

I hope this helps! Thank you.

Answer 2

I guess that fine tuning through Azure is just not a feasible solution for us then. In our use case we would like to have multiple finetuned models used infrequently and would therefore much prefer the OpenAI price structure where you pay a premium on the usage instead of hosting costs. Paying 15.000 of dollars pr year per model is totally out of scope for us.