Query ingested documents using the Inference API

Article
11/18/2024

Important

This is a preview feature. This information relates to a prerelease feature that may be substantially modified before it's released. Microsoft makes no warranties, expressed or implied, with respect to the information provided here.

To query the corpus after the documents are configured and ingested, provide the following details:

Authentication

You need to add authorization header to your request. Further, you need to get a token for your application. See the following example to know how to generate a token in the sample. 

{'Content-Type': 'application/json', 'Authorization': 'Bearer {token}'}

Ask API Top Level 

Go to the deployed resource group, and find api_management_name.

https://{api_management_name}.azure-api.net/inference/v1/Ask

Request fields

The request contains the following top-level fields:

Field Name	Type	Required	Description/Example
query	string	Yes	Example: When is Campbell Soup releasing earnings?
filter	string	No	A web page to learn about filtering Azure AI Search. https://learn.microsoft.com/azure/search/search-filters
systemMessage	string	No	Your system message. Note: This system message replaces our standard system message and might affect performance. We don't recommend using it.

Response fields

Field Name	Type	Description
message	string	The result of the query
citationsResult	CitationsResult object	A citation within the message that points to a specific quote from a file associated with the assistant or the message.

CitationsResult object

Field Name	Type	Description
citations	Dictionary. Key: string, value: Citation object	Key represents the document, for example, doc1.
intent	Array of Strings	Recognized intentions.

Citation object

Field Name	Type	Description
content	list of chunks	The returned chunk content
title	string	The title of the document
url	String	The URL to the document
fileName	String	The file name
pageNumber	Int	The page number of the chunk
chunkId	String	The chunk ID

Possible filters

Field Name	Type	Comments
filepath	String	`filepath eq 'morningstar ford report.pdf'`
page_number	Int32
contains_image	Boolean
title	String
document_metadata/author	String
document_metadata/summary	String
document_metadata/keywords	String
document_metadata/creation_date	DateTimeOffset	Use this field to filter by certain criteria, for example, the last three months: `document_metadata/creation_date ge 2023-12-19T00:00:00Z`
document_metadata/title	String
document_metadata/additional_metadata/financial_quarter	String
document_metadata/additional_metadata/financial_year	String
document_metadata/additional_metadata/organization	String

Test the API

You can test the Inference API using the Azure API Management service on the Azure portal. You need to use the API endpoint in your code to query the data. The following steps explain the entire process of testing the API:

Note

The API Management Gateway URL is available on the API Management Overview page.

On the Azure portal, go to your resource group and select the API Management resource.

On the API Management resource, go to APIs > APIs > Inference > v1. You should see the Financial documents analysis ASK API.

Select Test > Post ASK.

Request an access token for the app registration that you created.

a. Use the Azure CLI and sign in to the tenant which includes the deployment, as a user with permissions to call the API.

Run the following command: az account get-access-token --resource <app-reg cli id>

Here, app-reg cli-id stands for the Application (client) ID.

You should receive a response similar to the following command output:
```
{ 
 "accessToken": "<JWT-Token>" 
 "expiresOn": "2024-11-11 16:13:11.000000", 
 "expires_on": 1731334391, 
 "subscription": "<subscription_id>", 
 "tenant": "<tenant_id>" 
 "tokenType": "Bearer" 
}
```
b. To access the API, include the JWT-token in the request. Copy the JWT-token from the az command response, and add it as the authorization header in the request. The value of the header should be Bearer <JWT-Token>. Change the query value to Hi and select Send.

You should receive a response similar to the following HTTP message:

c. To test the API, change the query value to a text string that might lead to a response based on the content of the ingested documents, and select Send.
Test the process of making the API call directly from your code. The following Python code snippet shows how you can use the API in your code.

from azure.identity import DefaultAzureCredential 
import requests 
# Get Credentials 
credentials = DefaultAzureCredential() 
# Get the access token 
access_token = credentials.get_token("<App Registration Scope>",tenant_id="<your tenant id>") 
# Define the URL, headers, and payload 
url = "https://<api managment url>/inference/v1/Ask" 
headers = { 
    "Content-Type": "application/json", 
    "Authorization": f"Bearer {access_token.token}" 
} 
payload = { 
    "query": "<your query>" 
} 
# Execute the POST request 
response = requests.post(url, headers=headers, json=payload) 
# Print the response 
print(response.status_code) 
print(response.json())

Share via

Query ingested documents using the Inference API

Authentication