404 Resource Not Found using OpenAI Azure gpt-4

Question

Trying to use the OpenAI Azure client for the first time (switching from openai direct), but having trouble. Here's how i instantiate my client:

            return AsyncAzureOpenAI(
                api_key=os.getenv("AZURE_OPENAI_API_KEY"),  
                api_version="2024-01-25-preview",
                azure_deployment="NAME",
                azure_endpoint=azure_endpoint,
            )

(placeholder for deployment name)

Here's my actual deployment. I do pass in the correct azure_endpoint as well. I keep getting 404 resource not found when i try to call this. here's my code for how i use the client:

            async with get_client(model.azure) as client:
                if stream:
                    async for chunk in await client.chat.completions.create(**params):

Anyone know what's going on? Thanks

Answer 1

Found the answer. The problem was actually in the request params. Normally, openai expects a model name to be passed in for model like

gpt-4-1106-preview

But in the case of azure, it expects the given model name in your deployment, so whatever name you wrote down for the deployment should be passed in here.

    params = {
        "model": model.name, // NAME IN AZURE
        "messages": chat_messages,
        "max_tokens": model_max_tokens,
        "temperature": temperature,
        "stream": stream,
    }

Answer 2

@Randy Song

Thanks for reaching out to us and sharing the workaround for this issue. Please let us know if you have any other issues. Please feel free to open a new thread if needed.

I will remove the duplicate posts.

Thanks again.

Regards,

Yutong

Answer 3

To anyone else finding this issue later, I had an issue with always getting a "HTTP 404 Resource not found" error specifically after upgrading to the "Azure.AI.OpenAI" Nuget package version "2.0.0-beta.5". In the new version of the Nuget package, I was trying to outright create a new AssistantClient and passing in my endpoint and API key. But apparently, you have to first create an AzureOpenAIClient , then create an AssistantClient (or whatever other client you need), like this:

var azureOpenAIClient = new AzureOpenAIClient(myEndpointUri, myApiKey);
var assistantClient = azureOpenAIClient.GetAssistantClient();
var chatClient = azureOpenAIClient.GetChatClient("gpt-4o");