Guidance Needed for APIM SKU & VNET Integration in Shared Azure OpenAI Cluster

Question

I'm working on proposing a new shared cluster of Azure OpenAI services to better manage quota and allocation across our environments.

Current Setup:

• Each environment has its own resource group and VNET.
• All resources use private endpoints in the respective VNET, with public accessibility disabled.
• Users connect through an internal network via ExpressRoute links between on-prem and Azure.

This setup is working well overall.

Problem Statement: We face challenges managing quotas and capacity for Azure OpenAI services across different environments.

Proposed Solution: I want to introduce a new VNET that hosts the Azure OpenAI services and Azure API Management (APIM). Each environment would then connect to the OpenAI services through APIM, which will allow us to utilize load balancing, throttling, and prioritization at the APIM policy level.

Challenges: I'm struggling with deciding on the appropriate APIM SKU for VNET integration. Here are some key requirements:

• All Azure OpenAI services in the new VNET should have private endpoints.
• APIM should be able to connect to these private endpoints.
• All environment-specific VNETs need to be able to reach APIM.

Simplified Diagram

Questions:

• Which APIM instance SKU should I choose? It seems like Premium is the answer, but I'm unsure, especially considering the v2 SKUs.
• How can I ensure that the environment VNETs can connect to APIM in the shared VNET? Would VNET peering be sufficient, or do I need to create private endpoints for APIM in the environment-specific VNETs?
• Will APIM be able to connect to the Azure OpenAI services using private endpoints effectively?
• Any insights or recommendations on the proper APIM SKU for this use case, and the best way to achieve cross-VNET communication, would be greatly appreciated.

Thanks in advance!

Answer 1

Hi @Jerzy Czopek Greetings! Thank you for posting this question on this forum.

Which APIM instance SKU should I choose

Based on the scenario you provided, the appropriate APIM instance SKU for VNET integration would be the Premium SKU. The Premium SKU provides advanced features such as multi-region deployment, autoscaling, and more advanced security features, which would aid in managing quotas and capacity for Azure OpenAI services across different environments. Please refer the documentation Automatically scale an Azure API Management instance which provides details on how you can implement Auto scaling for the APIM instance

How can I ensure that the environment VNETs can connect to APIM in the shared VNET?

Vnet peering should be enough to let APIM access the configured end points. Please take a look into the article Use a virtual network to secure inbound or outbound traffic for Azure API Management which provided different details on different approaches you can take to implement a Vnet with APIM as per your needs.

Will APIM be able to connect to the Azure OpenAI services using private endpoints effectively?

APIM uses a private IP address for a connection in the VNet or a peered VNet. Please refer the article section Routing for additional details on this.

The article Virtual network peering provides good pointer on Vnet peering can be achieved.

Hope this helps! Please let us know if you have any additional questions or need further assistance.