Unable to create fine-tuning job with gpt-4o in East US 2 Region

Question

Hi,

I'm trying to fine-tune a gpt-4o model which is deployed in East US 2, which according to the documentation should be supported. However, I'm getting this error:

Error code: 400 - {'error': {'code': 'invalidPayload', 'message': 'The specified base model does not support fine-tuning.'}}

Here's the code I'm using:

client = openai.AzureOpenAI(
    azure_endpoint=f"https://{east-us-2-resource}.openai.azure.com",
    api_key=API_KEY,
    api_version="2024-12-01-preview"
)


response = client.files.create(
    file=open(FILENAME, "rb"),
    purpose='fine-tune'
)

file_id = response.id

resp_create = client.fine_tuning.jobs.create(
                training_file=file_id,
                model='gpt-4o'
            )

The same code with gpt-35-turbo in the same resource/region succeeds. And I know that gpt-4o-mini is not available to fine-tune in eastus2, but gpt-4o is supposed to be.

Am I missing something?

Answer 1

Hello Ching-Wei,

According to @Ching-Wei

Thank you to @Sina Salam for your suggestions. The answer is related to your suggestion 3.

Issue: The model customer deployed was the wrong version. I missed the fact that the docs specify only specific model versions that support fine-tuning - for gpt-4o it is only gpt-4o-2024-08-06. My deployment had version 2024-11-20, which is the latest version, but apparently doesn't support fine-tuning.

Error Message: Error code: 400 - {'error': {'code': 'invalidPayload', 'message': 'The specified base model does not support fine-tuning.'}}

Solution: According to Customer "When calling the fine-tuning method for gpt-4o, you have to use the full model string with version, in this case gpt-4o-2024-08-06. Using just the model's name gives an error that the base model doesn't support fine-tuning. BTW this is not true for gpt-35-turbo or gpt-4o-mini - when calling fine-tuning with those model strings alone (no versions) the fine-tuning job will be created no problem."

I hope this is helpful! Do not hesitate to let me know if you have any other questions or clarifications.

---

Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful.

Answer 2

Hello Ching-Wei,

Welcome to the Microsoft Q&A and thank you for posting your questions here.

I understand that you are unable to create fine-tuning job with gpt-4o in East US 2 Region.

1. Use the latest preview API version is 2025-01-01-preview to ensures access to the most recent features and updates.
2. Confirm that your Azure OpenAI resource is deployed in the East US 2 region and has fine-tuning capabilities enabled. If fine-tuning isn't enabled, you may need to request access or adjust your resource settings. You can enable it if you Go to your Azure OpenAI resource > check the Overview section to ensure your resource is deployed in the East US 2 region > in the Settings section, look for Fine-tuning options. If fine-tuning is not enabled, you may see an option to request access or enable it.
3. When creating the fine-tuning job, use the exact model identifier: gpt-4o-2024-08-06. This ensures the system recognizes the model for fine-tuning. https://learn.microsoft.com/en-us/azure/ai-services/openai/tutorials/fine-tune and https://community.openai.com/t/cannot-see-failure-error-message-after-trying-to-run-a-fine-tuning-job-on-gpt-4o-2024-08-06/925141
4. Also, make sure your training data is in the correct JSON Lines (.jsonl) format, structured as follows:

      {"messages": [
          {"role": "system", "content": "System prompt here."},
          {"role": "user", "content": "User input here."},
          {"role": "assistant", "content": "Expected response here."}
      ]}
   

   Proper formatting is crucial to avoid payload errors during the fine-tuning process.
5. Implement the Fine-Tuning process like the below step in the code to fine-tune the GPT-4o model:

      import openai
      import os
      # Initialize the Azure OpenAI client
      client = openai.AzureOpenAI(
          azure_endpoint="https://<your-resource-name>.openai.azure.com",
          api_key=os.getenv("AZURE_OPENAI_API_KEY"),
          api_version="2025-01-01-preview"
      )
      # Upload the training file
      training_response = client.files.create(
          file=open("training_data.jsonl", "rb"),
          purpose="fine-tune"
      )
      training_file_id = training_response.id
      # Create the fine-tuning job
      fine_tune_response = client.fine_tuning.jobs.create(
          training_file=training_file_id,
          model="gpt-4o-2024-08-06"
      )
      fine_tune_job_id = fine_tune_response.id
      # Monitor the fine-tuning job status
      import time
      while True:
          status_response = client.fine_tuning.jobs.retrieve(fine_tune_job_id)
          status = status_response["status"]
          if status in ["succeeded", "failed"]:
              break
          print(f"Fine-tuning job status: {status}")
          time.sleep(60)  # Wait for 1 minute before checking again
      # Deploy the fine-tuned model
      if status == "succeeded":
          fine_tuned_model = status_response["fine_tuned_model"]
          deployment_response = client.deployments.create(
              model=fine_tuned_model,
              deployment_name="fine-tuned-gpt-4o",
              scale_settings={"capacity": 1}
          )
          print("Deployment successful.")
      else:
          print("Fine-tuning job failed.")
   

I hope this is helpful! Do not hesitate to let me know if you have any other questions or clarifications.

---

Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful.