How can I know when I can use a certain version of the GPT model in Azure?

Zeyang Zhou 0 Microsoft Employee

I'd like to use gpt4o-audio-preview, but I can't find it in my deployment model. Can I get the timeline for when I can use it?

Zeyang Zhou 0 Reputation points Microsoft Employee

2024-11-29T08:26:29.6733333+00:00

like this:

https://platform.openai.com/docs/guides/audio?lang=curl&audio-generation-quickstart-example=audio-in

and

https://community.openai.com/t/how-can-i-pass-a-system-prompt-and-audio-user-input-to-get-a-text-output-back/1002483/10

That I'd like can input prompt and audiothat let GPT help me directly analysis audio, not audio to text then analysis text. e.g. tell me how long is the audio, how many speaker in the audio, what's the speaker's tone....

1 answer

santoshkc 10,475 Reputation points Microsoft Vendor

2024-11-29T10:07:50.2833333+00:00

Hi @Zeyang Zhou,

Thank you for your query regarding the availability of the gpt4o-audio-preview model. Azure OpenAI Services release new models gradually, and their availability depends on region and account eligibility. Currently, this model is available for deployment in the East US 2 and Sweden Central regions with API version 2024-10-01-preview. However, you can keep an eye on the Azure OpenAI Service release notes for updates on What's new in Azure OpenAI Service page will be updated accordingly with more details.

Azure does not yet support direct audio analysis like speaker count or tone detection natively with GPT models. You can use Azure Speech-to-Text for transcription and metadata extraction, then pass this information (e.g., speaker count, duration, tone) to GPT for advanced reasoning and analysis.

I hope you understand. And, if you have any further query do let us know.

If this answers your query, do click Accept Answer and Yes for was this answer helpful.
Please sign in to rate this answer.
Zeyang Zhou 0 Reputation points Microsoft Employee

2024-12-02T05:50:36.7133333+00:00

Thanks, I not found gpt4o-audio-preview in deployment candidate, but found gpt-4o-realtime-preview is that means gpt-4o-realtime-preview at least contains ability of gpt4o-audio-preview?

About Azure Speech-to-Text is directly analysis audio, that not tanscription first and hen analysis text? Is that Whisper or other things, could I deploy it in Azure AI Foundry, or any document can I reference how to use?

santoshkc 10,475 Reputation points Microsoft Vendor

2024-12-03T14:23:19.8633333+00:00

Hi @Zeyang Zhou,

Thank you for your follow-up query.

The gpt-4o-realtime-preview model supports real-time audio processing, but it doesn't fully contain the features of gpt4o-audio-preview. It's designed for low-latency conversational interactions, like support agents or assistants.

Azure Speech-to-Text transcribes audio to text before further analysis, such as speaker identification or tone. For direct audio analysis (e.g., detecting tone or counting speakers without transcription), Azure doesn't currently support this natively.

Whisper can transcribe audio to text and is available for use in Azure. For more details, check: What is the Whisper model? and Speech to text with the Azure OpenAI Whisper model.

I hope this helps. Thank you.

Zeyang Zhou 0 Reputation points Microsoft Employee

2024-12-04T00:23:01.6166667+00:00

I tried gpt-4o-realtime-preview, it has ablitiy to analysis audio directly, but performance not reach I expect.

In Open AI wiki, they mentioned:

How is audio in Chat Completions different from the Realtime API?

The underlying GPT-4o audio model is exactly the same. The Realtime API operates the same model at lower latency.

I'm not sure weather gpt-4o-audio-preview will have better performance.

From AzureOpenAI page, gpt-4o-realtime-preview avaliable in Oct, but no any related message about gpt-4o-audio-preview.

I tried Whisper, and it's great for transcribing audio to text, but I couldn't find a way to use it for analyzing the audio itself.

About Speech-to-Text, seems it mostly focus on transcription not analysis audo quality.

I'd like can use gpt-4o-audio-preview, and API like this, any timeline that Azure AI can support gpt-4o-audio-preview and the API?

https://community.openai.com/t/how-can-i-pass-a-system-prompt-and-audio-user-input-to-get-a-text-output-back/1002483/9

import base64 from openai import OpenAI client = OpenAI() def read_audio_file(filepath: str): with open(filepath, "rb") as audio_file: return audio_file.read() # Read and encode the audio data as base64 audio_data = read_audio_file("PATH_TO_AUDIO_INPUT.WAV") encoded_string = base64.b64encode(audio_data).decode("utf-8") # Send the encoded audio as base64 string in the `input_audio` completion = client.chat.completions.create( model="gpt-4o-audio-preview", modalities=["text"], messages=[ { "role": "system", "content": "Your job is to transcribe user audio and tell the general tone it is in", }, { "role": "user", "content": [ { "type": "text", "text": "What is in this recording?" }, { "type": "input_audio", "input_audio": { "data": encoded_string, # Use the base64 string here "format": "wav", }, } ], }, ], ) # Print the response print(completion.choices.message.content)
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

How can I know when I can use a certain version of the GPT model in Azure?

1 answer

Your answer