I cannot register "metadata" info in Azure OpenAI Batch API

TERADA, Tomohiko (寺田智彦) 20

Since a couple days ago, I've not able to register "metadata" info when I create new batch Job.

Is there anyone who have same issue?

I call 'create new batch job' API with my local data in "metadata".

I receive the job has been registered successfully with my "metadata".

However, if I call 'list job batch', my metadata is empty.

It's been working properly until Sep-4.

Regards,

romungi-MSFT 45,036 Reputation points Microsoft Employee

2024-09-06T14:01:13.71+00:00

@TERADA, Tomohiko (寺田智彦) Are you referring to your deployments of batch endpoints through Azure OpenAI studio or REST API? I have referred the official documentation of Azure OpenAI Batch API and the create batch job does not mention the availability of metadata in the reference, see here for details.

I have also tried the Azure OpenAI studio and see the same behavior where there is no option to add metadata in request body.

I have tried to add the metadata through REST API requests during create but the list or GET batch API does not provide the added metadata in the response, it is always set to null.

I think for the Azure OpenAI implementation metadata option might currently be turned off.

If you have set metadata in previous jobs before 4th Sep, do the GET provide this data now? Thanks!!
TERADA, Tomohiko (寺田智彦) 20 Reputation points

2024-09-06T20:36:17.0166667+00:00

@romungi-MSFT ,

Thank you for your comment.

I'm using REST API

If you have set metadata in previous jobs before 4th Sep, do the GET provide this data now?

Yes, I still can get my metadata which I stored before 4th Sep.

I have referred the official documentation of Azure OpenAI Batch API and the create batch job does not mention the availability of metadata in the reference, see here for details.

The document mentions batch object and there is a metadata property.

TERADA, Tomohiko (寺田智彦) 20

I call this and I get my metadata which I stored before 4th Sep.

job_list = client.batches.list(limit=100) print(job_list)

Batch(id='batch_c04f8856-2d44-45f7-8aa9-7cb6bc1625fa', completion_window='24h', created_at=1724118513, endpoint='/chat/completions', input_file_id='file-bd85d9d92940430daccfb09497cb39c5', object='batch', status='completed', cancelled_at=None, cancelling_at=None, completed_at=1724119083, error_file_id='file-9a4edd76-53f1-4a75-94aa-90ab2fefbbad', errors=None, expired_at=None, expires_at=1724204913, failed_at=None, finalizing_at=1724118982, in_progress_at=1724118670, metadata={'user_email': 'tomohiko.g1s8.terada@xxxx', 'data_file': 'xxxxx.xlsx', 'comment': '', 'temperature': '0.7', 'top_p': '1.0', 'frequency_penalty': '0.0', 'presence_penalty': '0.0'}, output_file_id='file-a1046cb4-569b-4504-85cb-03932f07bb57', request_counts=BatchRequestCounts(completed=504, failed=0, total=504))

romungi-MSFT 45,036 Reputation points Microsoft Employee

2024-09-09T07:30:54.0733333+00:00

@TERADA, Tomohiko (寺田智彦) Thanks for sharing these details. Let me check internally to find out if this feature is working as expected. Thanks!!
Emily 5 Reputation points

2024-09-12T12:55:38.9966667+00:00
@romungi-MSFT I'd like to add that I've just encountered this exact problem. I can retrieve metadata for old batches, but not new ones.

Reproduction steps:

created_batch = batch_client.batches.create( input_file_id=batch_input_file_id, endpoint="/chat/completions", completion_window="24h", metadata={ "description": "some description", } )

created_batch.metadata contains the description, as expected.

On the very next line, I call:

all_batches = batch_client.batches.list()

all_batches.data[0] is the batch I just created (its input_file_id is equal to batch_input_file_id), but its metadata is None
TERADA, Tomohiko (寺田智彦) 20 Reputation points

2024-09-12T13:43:22.8066667+00:00

Still, I cannot retrieve metadata.
So, I switched to OpenAI (not Azure), and now my program works well.
Why did Microsoft make it so that only Azure OpenAI cannot use metadata...?

Accepted answer

romungi-MSFT 45,036 Reputation points Microsoft Employee

2024-09-17T11:42:57.86+00:00
@TERADA, Tomohiko (寺田智彦) I can confirm that there has been a bug with batch jobs that caused the metadata property to be null. A fix has been deployed now and I can see metadata property returns with the data added in the create job. Please see a response of my batch job that was submitted today which contains the metadata.

{ "cancelled_at": null, "cancelling_at": null, "completed_at": 1726573123, "completion_window": "24h", "created_at": 1726572674, "error_file_id": "file-4ec49c3f-10e2-429d-aefd-f5b2e38998c5", "expired_at": null, "expires_at": 1726659073, "failed_at": null, "finalizing_at": 1726573054, "id": "batch_033b4cc2-d00c-4df9-967e-fccde36e6578", "in_progress_at": 1726572839, "input_file_id": "file-380cd850c29947b695bca88a7006bcae", "errors": null, "metadata": { "modified": "true", "user": "abc123" }, "object": "batch", "output_file_id": "file-bbb36a01-497e-42c8-88a7-b2e68ecef292", "request_counts": { "total": 3, "completed": 3, "failed": 0 }, "status": "completed", "endpoint": "/chat/completions" }

Could you please retry with a new job and check if it is working as expected for you. Thanks!!

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Please sign in to rate this answer.

1 person found this answer helpful.
TERADA, Tomohiko (寺田智彦) 20 Reputation points

2024-09-17T13:27:19.09+00:00

@romungi-MSFT ,

Thank you for your support. Now, it works.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

I cannot register "metadata" info in Azure OpenAI Batch API

0 additional answers

Your answer