Plan and manage costs for Azure AI Foundry

Article
11/21/2024

Important

Items marked (preview) in this article are currently in public preview. This preview is provided without a service-level agreement, and we don't recommend it for production workloads. Certain features might not be supported or might have constrained capabilities. For more information, see Supplemental Terms of Use for Microsoft Azure Previews.

This article describes how you plan for and manage costs for Azure AI Foundry. First, you use the Azure pricing calculator to help plan for Azure AI Foundry costs before you add any resources for the service to estimate costs. Next, as you add Azure resources, review the estimated costs.

Tip

Azure AI Foundry does not have a specific page in the Azure pricing calculator. Azure AI Foundry is composed of several other Azure services, some of which are optional. This article provides information on using the pricing calculator to estimate costs for these services.

You use Azure AI services in Azure AI Foundry portal. Costs for Azure AI services are only a portion of the monthly costs in your Azure bill. You're billed for all Azure services and resources used in your Azure subscription, including the third-party services.

Prerequisites

Cost analysis in Microsoft Cost Management supports most Azure account types, but not all of them. To view the full list of supported account types, see Understand Cost Management data. To view cost data, you need at least read access for an Azure account. For information about assigning access to Microsoft Cost Management data, see Assign access to data.

Estimate costs before using Azure AI services

Use the Azure pricing calculator to estimate costs before you add Azure AI services.

Select a product such as Azure OpenAI in the Azure pricing calculator.
Enter the number of units you plan to use. For example, enter the number of tokens for prompts and completions.
You can select more than one product to estimate costs for multiple products. For example, select Virtual Machines to add potential costs for compute resources.

As you add new resources to your project, return to this calculator and add the same resource here to update your cost estimates.

Costs that typically accrue with Azure AI Foundry

When you create resources for a hub, resources for other Azure services are also created. They are:

Service pricing page	Description with example use cases
Azure AI services	You pay to use services such as Azure OpenAI, Speech, Content Safety, Vision, Document Intelligence, and Language. Costs vary for each service and for some features within each service. For more information about provisioning of Azure AI services, see Azure AI Foundry hubs.
Azure AI Search	An example use case is to store data in a vector search index.
Azure Machine Learning	Compute instances are needed to run Visual Studio Code (Web or Desktop) and prompt flow via Azure AI Foundry. When you create a compute instance, the virtual machine (VM) stays on so it's available for your work. Enable idle shutdown to save on cost when the VM is idle for a specified time period. Or set up a schedule to automatically start and stop the compute instance to save cost when you aren't planning to use it.
Azure Virtual Machine	Azure Virtual Machines gives you the flexibility of virtualization for a wide range of computing solutions with support for Linux, Windows Server, SQL Server, Oracle, IBM, SAP, and more.
Azure Container Registry Basic account	Provides storage of private Docker container images, enabling fast, scalable retrieval, and network-close deployment of container workloads on Azure.
Azure Blob Storage	Can be used to store Azure AI Foundry project files.
Key Vault	A key vault for storing secrets.
Azure Private Link	Azure Private Link enables you to access Azure PaaS Services (for example, Azure Storage and SQL Database) over a private endpoint in your virtual network.

Costs might accrue before resource deletion

Before you delete a hub in the Azure portal or with Azure CLI, the following sub resources are common costs that accumulate even when you aren't actively working in the workspace. If you're planning on returning to your hub at a later time, these resources might continue to accrue costs:

Azure AI Search (for the data)
Virtual machines
Load Balancer
Azure Virtual Network
Bandwidth

Each VM is billed per hour it's running. Cost depends on VM specifications. VMs that are running but not actively working on a dataset are still charged via the load balancer. For each compute instance, one load balancer is billed per day. Every 50 nodes of a compute cluster have one standard load balancer billed. Each load balancer is billed around $0.33/day. To avoid load balancer costs on stopped compute instances and compute clusters, delete the compute resource.

Compute instances also incur P10 disk costs even in stopped state. This cost is because any user content saved to disk is persisted across the stopped state similar to Azure VMs. We're working on making the OS disk size/ type configurable to better control costs. For Azure Virtual Networks, one virtual network is billed per subscription and per region. Virtual networks can't span regions or subscriptions. Setting up private endpoints in virtual network setups might also incur charges. If your virtual network uses an Azure Firewall, the firewall might also incur charges. Bandwidth usage is charged; the more data transferred, the more you're charged.

Tip

Using a managed virtual network is free. However some features of the managed network rely on Azure Private Link (for private endpoints) and Azure Firewall (for FQDN rules) and will incur charges. For more information, see Managed virtual network isolation.

Costs might accrue after resource deletion

After you delete a hub in the Azure portal or with Azure CLI, the following resources continue to exist. They continue to accrue costs until you delete them.

Azure Container Registry
Azure Blob Storage
Key Vault
Application Insights (if you enabled it for your hub)

Monitor costs

As you use Azure AI Foundry with hubs, you incur costs. Azure resource usage unit costs vary by time intervals (seconds, minutes, hours, and days) or by unit usage (bytes, megabytes, and so on). You can see the incurred costs in cost analysis.

When you use cost analysis, you view hub costs in graphs and tables for different time intervals. Some examples are by day, current and prior month, and year. You also view costs against budgets and forecasted costs. Switching to longer views over time can help you identify spending trends. And you see where overspending might occur. If you create budgets, you can also easily see where they're exceeded.

Monitor Azure AI Foundry project costs

You can get to cost analysis from the Azure portal. You can also get to cost analysis from the Azure AI Foundry.

Important

Your AI Foundry project costs are only a subset of your overall application or solution costs. You need to monitor costs for all Azure resources used in your application or solution. For more information, see Azure AI Foundry hubs.

For the examples in this section, assume that all Azure AI Foundry resources are in the same resource group. But you can have resources in different resource groups. For example, your Azure AI Search resource might be in a different resource group than your project.

Here's an example of how to monitor costs for a project. The costs are used as an example only. Your costs vary depending on the services that you use and the amount of usage.

Sign in to Azure AI Foundry.
Select your project and then select Management center from the left menu.
Under the Project heading, select Overview.
Select View cost for resources from the Total cost section. The Azure portal opens to the resource group for your project.
Expand the Resource column to see the costs for each service that's underlying your project. But this view doesn't include costs for all resources that you use in a project.
Select Costs by resource > Resources.
On the Cost analysis page where you're taken to, make sure the scope is set to your resource group.

In this example:
- The resource group name is rg-contosoairesource.
- The total cost for all resources and services in the resource group is $222.97. In this example, $222.97 is the total cost for your application or solution that you're building with Azure AI Foundry. Again, this example assumes that all Azure AI Foundry resources are in the same resource group. But you can have resources in different resource groups.
- The project name is contoso-outdoor-proj.
- The costs that are limited to resources and services in the project total $212.06.
Expand contoso-outdoor-proj to see the costs for services underlying the project resource.
Expand contoso_ai_resource to see the costs for services underlying the hub resource. You can also apply a filter to focus on other costs in your resource group.

You can also view resource group costs directly from the Azure portal. To do so:

Sign in to Azure portal.
Select Resource groups.
Find and select the resource group that contains your Azure AI Foundry resources.
From the left navigation menu, select Cost analysis.

For more information, see the Azure pricing calculator.

Monitor costs for models offered through the Azure Marketplace

Models deployed as a service using pay-as-you-go are offered through the Azure Marketplace. The model publishers might apply different costs depending on the offering. Each project in Azure AI Foundry portal has its own subscription with the offering, which allows you to monitor the costs and the consumption happening on that project. Use Microsoft Cost Management to monitor the costs:

Sign in to Azure portal.
On the left navigation area, select Cost Management + Billing and then, on the same menu, select Cost Management.
On the left navigation area, under the section Cost Management, select now Cost Analysis.
Select a view such as Resources. The cost associated with each resource is displayed.
On the Type column, select the filter icon to filter all the resources of type microsoft.saas/resources. This type corresponds to resources created from offers from the Azure Marketplace. For convenience, you can filter by resource types containing the string SaaS.
One resource is displayed for each model offer per project. Naming of those resources is [Model offer name]-[GUID].
Select to expand the resource details to get access to each of the costs meters associated with the resource.
- Tier represents the offering.
- Product is the specific product inside the offering.
Some model providers might use the same name for both.

Tip

Remember that one resource is created per each project, per each plan your project subscribes to.

When you expand the details, costs are reported per each of the meters associated with the offering. Each meter might track different sources of costs like inferencing, or fine tuning. The following meters are displayed (when some cost is associated with them):

Meter	Group	Description
`paygo-inference-input-tokens`	Base model	Costs associated with the tokens used as input for inference of a base model.
`paygo-inference-output-tokens`	Base model	Costs associated with the tokens generated as output for the inference of base model.
`paygo-finetuned-model-inference-hosting`	Fine-tuned model	Costs associated with the hosting of an inference endpoint for a fine-tuned model. This value isn't the cost of hosting the model, but the cost of having an endpoint serving it.
`paygo-finetuned-model-inference-input-tokens`	Fine-tuned model	Costs associated with the tokens used as input for inference of a fine tuned model.
`paygo-finetuned-model-inference-output-tokens`	Fine-tuned model	Costs associated with the tokens generated as output for the inference of a fine tuned model.

Create budgets

You can create budgets to manage costs and create alerts that automatically notify stakeholders of spending anomalies and overspending risks. Alerts are based on spending compared to budget and cost thresholds. Budgets and alerts are created for Azure subscriptions and resource groups, so they're useful as part of an overall cost monitoring strategy.

Budgets can be created with filters for specific resources or services in Azure if you want more granularity present in your monitoring. Filters help ensure that you don't accidentally create new resources that cost you more money. For more about the filter options when you create a budget, see Group and filter options.

Export cost data

You can also export your cost data to a storage account. Exporting data is helpful when you or others need to do more data analysis for costs. For example, finance teams can analyze the data using Excel or Power BI. You can export your costs on a daily, weekly, or monthly schedule and set a custom date range. Exporting cost data is the recommended way to retrieve cost datasets.

Understand the full billing model for Azure AI services

Azure AI services run on Azure infrastructure that accrues costs along with Azure AI when you deploy the new resource. It's important to understand that extra infrastructure might accrue cost. You need to manage that cost when you make changes to deployed resources.

When you create or use Azure AI services resources, you might get charged based on the services that you use. There are two billing models available for Azure AI services:

Pay-as-you-go: Pay-as-you-go pricing, you're billed according to the Azure AI services offering that you use, based on its billing information.
Commitment tiers: With commitment tier pricing, you commit to using several service features for a fixed fee, enabling you to have a predictable total cost based on the needs of your workload. You're billed according to the plan you choose. See Quickstart: purchase commitment tier pricing for information on available services, how to sign up, and considerations when purchasing a plan.

Note

If you use the resource above the quota provided by the commitment plan, you will be charged for the additional usage as per the overage amount mentioned in the Azure portal when you purchase a commitment plan.

You can pay for Azure AI services charges with your Azure Prepayment (previously called monetary commitment) credit. However, you can't use Azure Prepayment credit to pay for charges for third-party products and services including ones from the Azure Marketplace.