Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
3,828 questions
what is Azure OpenAI API endpoint to train GPT 4o model programmatically using Direct Preference Optimization (DPO) Fine-Tuning
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(278.4, 25%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EM%3C/text%3E%3C/svg%3E)
MR
20
Reputation points
Hello,
Azure Open AI service provides the Direct Preference Optimization (DPO) fine tuning option in Azure Foundry.
We want to train the model programmatically but the Azure Open AI API endpoint doesn't have the request parameter for method type.
• Azure OpenAI API:
The current implementation does not allow specifying the type of fine-tuning:
curl -X POST $AZURE_OPENAI_ENDPOINT/openai/fine_tuning/jobs?api-version=2023-12-01-preview \
-H "Content-Type: application/json" \
-H "api-key: $AZURE_OPENAI_API_KEY" \
-d '{
"model": "gpt-35-turbo-0613.ft-5fd1918ee65d4cd38a5dcf6835066ed7",
"training_file": "<TRAINING_FILE_ID>",
"validation_file": "<VALIDATION_FILE_ID>",
"suffix": "<additional text used to help identify fine-tuned models>"
}'
• OpenAI API:
In contrast, the OpenAI fine-tuning API provides a clear mechanism to specify the type of fine-tuning:
const job = await openai.fineTuning.jobs.create({
training_file: "file-all-about-the-weather",
model: "gpt-4o-2024-08-06",
method: {
type: "dpo",
dpo: {
hyperparameters: { beta: 0.1 },
},
},
});
Is there any other Azure OpenAI API endpoint available to be able to specify the method type for fine tuning? Or when will this open be available in Azure OpenAI API?
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(281.6, 88%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESA%3C/text%3E%3C/svg%3E)
Saideep Anchuri 4,620 Reputation points Microsoft External Staff2025-02-05T05:16:05.0333333+00:00 Hi MR
Welcome to Microsoft Q&A Forum, thank you for posting your query here!
DPO is supported on GPT 4o models for now. You can change customization to "Direct preference optimization" in UI as we could not find any customization param in rest Api documentation.
kindly refer below link: model-support
Thank You.
-
MR 20 Reputation points
2025-02-05T14:47:23.6166667+00:00 Thank you, Saideep for reply. Yes, the Fine Tuning is available from UI using DPO method in Azure Foundry.
I would like to know when will this functionality be available in Azure OpenAI API ?
-
Saideep Anchuri 4,620 Reputation points Microsoft External Staff
2025-02-06T02:04:28.7233333+00:00 Hi MR
In January 2025, the latest updates for the Azure OpenAI API brought new features like GPT-4o-audio-preview and GPT-4o-realtime-preview. These updates enhance the API with advanced audio and speech capabilities, broadening the scope for AI applications in text, voice interactions, and audio analysis.so we recommend keeping an eye on the What's New in Azure OpenAI service for updates.
Thank You.
-
MR 20 Reputation points
2025-02-06T16:44:40.3833333+00:00 Hi Saideep,
Thank you. What's New in Azure OpenAI service is invalid URL showing label as "Bad Request". Please share valid URL.
If possible, can you please provide the information on plan for adding support for DPO as a fine-tuning method support in Azure OpenAI API?
Many Thanks.
-
Saideep Anchuri 4,620 Reputation points Microsoft External Staff
2025-02-06T16:57:32.98+00:00 Hi MR
Microsoft plans to implement this in the future, so we recommend keeping an eye on the for updates. whats-new
Azure OpenAI Service has introduced a public preview of Direct Preference Optimization (DPO) for fine-tuning models. DPO is a novel method for aligning large language models with human preferences, eliminating the requirement for a separate reward model typically needed in methods like Reinforcement Learning from Human Feedback (RLHF).
Regarding stability, DPO directly refines the model's behavior based on human preferences, thus sidestepping the instability issues commonly encountered when training and managing multiple models.
kindly refer below link: public-preview
Thank You.
-
Saideep Anchuri 4,620 Reputation points Microsoft External Staff
2025-02-07T03:56:08.0233333+00:00 Hi MR
We haven’t heard from you on the last response and was just checking back to see if the give response was helpful.
Thank You.
Sign in to comment
Sign in to answer