Programmatically upload file and get response as file or stream

Question

Hi,

We would like to leverage AI for a business case. We will upload recommended device configuration matrix as a file and add it to the DataSource and will create a search index.

Next there will be another file with current device configurations which could be old and outdated. This file will contain hundreds of thousands of rows.

Each row represents the current device configuration.

E.g.

Device-1,currenConfiguration-1,supportedDeviceConfiguration-1

Device-2,currenConfiguration-2,supportedDeviceConfiguration-2

...

Device-N,currenConfiguration-N,supportedDeviceConfiguration-N

We would like to feed this current device configuration file to AI service and get most appropriate recommendations from AI for each of the devices. AI has to provide the recommendation based on the configuration matrix file pre-uploaded in the DataSource.

We would like to get the response as a file / stream where each row will have the recommended configuration for the device.

e.g

Recommends Device-1 to use latestConfigration-1.1,latestSupportedDeviceConfiguration-1.1

Recommends Device-2 to use latestConfigration-2.1,latestSupportedDeviceConfiguration-2.1

...

Recommends Device-N to use latestConfigration-N.1,latestSupportedDeviceConfiguration-N.1

We would like to upload this file and get the response as a file/stream programmatically. We would like to know if this can be achieved via Java and if yes could you please point me to any APIs/Classes in the Azure AI Services Java SDK that we can use?

Answer 1

Hi, @Subijay Bhattacharya

You can check batch operations for completion model, where you need to upload file using file operations and make batch request passing file id got from uploading.

The file format should be of type .jsonl, create the .jsonl file with current device configuration file.

Example:

{"custom_id": "task-0", "method": "POST", "url": "/chat/completions", "body": {"model": "REPLACE-WITH-MODEL-DEPLOYMENT-NAME", "messages": [{"role": "system", "content": "<Give prompt to get required output for device 1>"}, {"role": "user", "content": "Your prompt for device config 1"}]}}

{"custom_id": "task-1", "method": "POST", "url": "/chat/completions", "body": {"model": "REPLACE-WITH-MODEL-DEPLOYMENT-NAME", "messages": [{"role": "system", "content": "<Give prompt to get required output for device 2>"}, {"role": "user", "content": "Your prompt for device config 2"}]}}

{"custom_id": "task-2", "method": "POST", "url": "/chat/completions", "body": {"model": "REPLACE-WITH-MODEL-DEPLOYMENT-NAME", "messages": [{"role": "system", "content": "<Give prompt to get required output for device 3>"}, {"role": "user", "content": "Your prompt for device config 1"}]}}

Like this you create for all the device configuration.

Below is the sample

        String fileName = "batch_tasks.jsonl";

        FileDetails fileDetails = new FileDetails(BinaryData.fromFile(openResourceFile(fileName)), fileName);

        OpenAIFile file = client.uploadFile(fileDetails, FilePurpose.BATCH);

        String fileId = file.getId();

        System.out.println("File uploaded, file ID = " + fileId);

        // Create a batch

        while (file.getStatus() == FileState.PENDING) {

            try {

                Thread.sleep(1000);

            } catch (InterruptedException e) {

                throw new RuntimeException(e);

            }

            file = client.getFile(file.getId());

        }

        Batch batch = client.createBatch(new BatchCreateRequest("/chat/completions", file.getId(), "24h"));

Next get output file content like below.

String outputFileId = batch.getOutputFileId();

        byte[] fileContent = client.getFileContent(outputFileId);

        System.out.println("File output ID is " + outputFileId);

        System.out.println("File content: ");

        System.out.println(new String(fileContent));

Refer this sample and more about batch support here.

Answer 2

Hi @Subijay Bhattacharya,

Thank you for reaching out to Microsoft Q&A forum!

You can achieve the described functionality using Azure AI services by leveraging the "Add Your Own Data" or "Assistant feature" from Azure OpenAI.

Programmatically upload the device configuration matrix into a vector store or azure blob storage and query the Azure OpenAI to get recommendations for each device based on the pre-uploaded data. To ensure the responses are formatted as required (e.g., "Recommends Device-1 to use latestConfiguration-1.1, latestSupportedDeviceConfiguration-1.1"), include this sample output in the chat prompt or system prompt during interaction.

Use Azure AI SDKs or APIs, such as the Java SDK for Azure OpenAI or AI Search, to upload data, process rows, retrieve recommendations, and compile results into a file or stream.

Please note: While Add Your Own Data is production-ready, but the Assistant feature is in public preview and not recommended for production use.

I hope this helps! Thank you.

Answer 3

Hi @Anonymous ,

I was getting few exceptions hence I tried the sample code in github.

With this I am still getting error. Typically the Batch process does not complete and the state is VALIDATING. After sometime it throws error. I added the below code to check the details

// Debug batch failure
if
System.
System.
batch.getErrors().getData().forEach(err -> System.
return
}

The error message says

Invalid deployment SKU 'GlobalStandard'. The deployment SKU needs to be 'Global-Batch' or 'DataZone-Batch' for batch API requests

I could not find where in my deployment I have to update the SKU. Could you please help?