url retured as null to HTTP POST

Question

Hi,

URL field in the POST HTTP request is returned null when I call on GPT 4o deployed in Azure AI Studio on my Azure AI search Index. This is despite the Index in Azure AI studio showing I've mapped url to a DocUrl ( custom field created to map to metadata_storage_path in the import and vectorize data section of Azure AI search):

Index Fields:

HTTP Request in Power Automate:

{
  "messages": @{variables('messagesArray')},
  "max_tokens": 4096,
  "temperature": 0.7,
  "top_p": 0.95,
  "frequency_penalty": 0,
  "presence_penalty": 0,
  "stream": false,
  "data_sources": [
    {
      "type": "azure_search",
      "parameters": {
        "endpoint": "https://**********.search.windows.net",
        "index_name": "brs-***-index",
        "semantic_configuration": "brs-***-index-semantic-configuration",
        "authentication": {
          "type": "api_key",
          "key": "********"
        },
        "embedding_dependency": {
          "type": "endpoint",
          "endpoint": "https://****/openai/deployments/text-embedding-ada-002/embeddings?api-version=2023-03-15-preview",
          "authentication": {
            "type": "api_key",
            "key": "****"
          }
        },
        "query_type": "vector_simple_hybrid",
        "in_scope": true,
        "role_information": "You are an AI assistant for the company '***' called '***™ AI' that helps people find information from *** support documents.",
        "strictness": 3,
        "top_n_documents": 5
      }
    }
  ]
}

Output from the HTTP request:

{
  "id": "d714a6d3-a9b1-4071-a5c7-**",
  "model": "gpt-4o",
  "created": 1728560139,
  "object": "extensions.chat.completion",
  "choices": [
    {
      "index": 0,
      "finish_reason": "stop",
      "message": {
        "role": "assistant",
        "content": "*** is a *** Intelligence Platform designed by ** to aid users in making informed decisions. .......",
        "end_turn": true,
        "context": {
          "citations": [
            {
              "content": "user acces is .............",
              "title": "User Interface.pdf",
              "url": null,
              "filepath": null,
              "chunk_id": "0"
            },
            {
              "content": "To login ..........."
              "title": "User Login.pdf",
              "url": null,
              "filepath": null,
              "chunk_id": "0"
            }
          ],
          "intent": ""
        }
      }
    }
  ],
  "usage": {
    "prompt_tokens": 5042,
    "completion_tokens": 323,
    "total_tokens": 5365
  },
  "system_fingerprint": "***"
}

Would really appreciate help here. @VenkateshDodda-MSFT

Thanks

Answer 1

Verify that the DocUrl field is correctly mapped to metadata_storage_path. The metadata_storage_path is typically where URLs are stored in Azure Cognitive Search. Verify that this field contains valid URLs for the documents in your index.

Verify also that the field DocUrl is correctly defined in your Azure Cognitive Search Index schema. It should be a searchable field, and the metadata_storage_path (or custom mapped field) should contain valid URLs pointing to the documents.

You can do this by reviewing the index configuration in the Azure Portal and checking whether the metadata_storage_path field is properly populated with URLs.

Before testing with GPT 4o, query the Azure Cognitive Search Index directly using the Azure Search REST API to verify that the DocUrl field is returning proper URLs. You can do this by sending a query to the search index:

   POST https://<your-search-service>.search.windows.net/indexes/<index-name>/docs/search?api-version=2020-06-30

   {

     "search": "*",

     "select": "DocUrl,metadata_storage_path",

     "top": 5

   }

If the URLs are returned correctly here, the issue is likely in how the mapping or retrieval is done in the GPT integration.

In your Power Automate flow, check that the variable messagesArray and the data_sources parameters are correctly formatted and are pulling the correct values from your index.

• The embedding_dependency.endpoint is pointing to a specific embedding model, but the URL field mapping might need to be ensured.
• Verify if url is being passed correctly in the Power Automate JSON payload.

Depending on the role you assigned to the AI assistant (role_information), ensure that it is not affecting the visibility or inclusion of the url field in the returned response. You can try reducing the strictness temporarily to check if that resolves the issue.

Since you're using a vector simple hybrid query (query_type: "vector_simple_hybrid"), ensure that the vector query process and relevance scoring are not filtering out or omitting the URL information from the results.

If none of the above steps resolve the issue, it may be helpful to add detailed logging around the point where the URL field is being processed or raise a support ticket with Azure, providing the context of the issue.

Let me know if you need help with any of these steps!

Answer 2

Hi @Viswavaageesh Chandramouli ,
Thanks for your Response!
Since you're making an HTTP request directly to GPT-4o using your Azure Cognitive Search index, and the issue lies in how GPT-4o is interpreting the DocUrl.
To ensure DocUrl is interpreted correctly in GPT-4o via an HTTP request:

• Structure the JSON Payload: Make sure DocUrl is explicitly mapped as url in the citations array, separate from content.

{
  "content": "Extracted document text from Azure Cognitive Search",
  "citations": [
    {
      "title": "User Interface.pdf",
      "url": "https://path-to-your-document",
      "filepath": null,
      "chunk_id": "0"
    }
  ]
}

• Consistent Naming: Ensure DocUrl from the search index is mapped directly to url under citations in the request.
• Test the Request: Send the structured payload to verify that GPT-4o recognizes url in citations as the distinct URL and not part of content.
  Let me know if you need further assistance .