How to config Azure OpenAI batch analysis?

Question

I need to analyze a large amount of documents with gpt-4o model, but I found that the batch models are more efficient in cost.
I used the Azure Openai batch deployments explained in this page:

*https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/batch?tabs=global-batch%2Cstandard-input%2Cpython-secure&pivots=rest-api
*
In this page it is explained how to config the file and the curl commands. I converted the curl commands to C# syntax.

First, I generated 2 item jsonl file but it took about 10 minutes to get the response file.

Then I created a 1000 item jsonl file to send to model, it failed! I have exactly set the operation window to 24h.
I want to know is there any extra config to get the responses in a suitable time?

Answer 1

Hi Elahe Dorani,
To optimize the response time and ensure successful processing of a larger file, here are a few suggestions:

• Ensure that you are not exceeding the batch size limits set by Azure OpenAI for processing. If the 1000-item jsonl file is too large for a single batch, consider splitting it into smaller batches and processing them sequentially.
• Make sure that the jsonl file format is optimized for processing efficiency. Check if there are any unnecessary elements or formatting that could be impacting the processing time.
• Double-check the configuration of your batch endpoint to ensure that it is optimized for handling larger batch sizes. You may need to adjust settings related to parallelization, resource allocation, or other parameters based on the size of the input data.

I hope this information helps.