How do we upload vector embeddings to an Azure AI Search index

Aravind Vijayaraghavan 0 Reputation points
2025-01-18T05:21:42.1733333+00:00

So I recently created vector fields and a vector profile for them and tried to mergeorUpload these vector embeddings to the index. I already embedded certain textual columns and stored them in a dataframe along with the rest of the columns in the same df. But when I run the code, I get this error regarding the vector fields:

Error 1)
An error occurred during document upload: () The request is invalid. Details: An unexpected 'PrimitiveValue' node was found when reading from the JSON reader. A 'StartArray' node was expected.

Error 2)
An error occurred during document upload: () The request is invalid. Details: Invalid JSON. A token was not recognized in the JSON content.

This is my code:

def upload_documents_to_search_client(df, chunk_size=32000):
    """Uploads documents to the search client in chunks."""
    data = [
        {
            "@search.action": "mergeOrUpload",
            "id": str(row["id"]),
            "vect_dev_exp_feedback": [] if pd.isna(row["vect_dev_exp_feedback"]) else row["vect_dev_exp_feedback"],
            "vect_neg_feedback": [] if pd.isna(row["vect_neg_feedback"]) else row["vect_neg_feedback"],  
            "vect_tools_feedback": [] if pd.isna(row["vect_tools_feedback"]) else row["vect_tools_feedback"],
            "vect_wlb_feedback": [] if pd.isna(row["vect_wlb_feedback"]) else row["vect_wlb_feedback"],
            "vect_growth_feedback": [] if pd.isna(row["vect_growth_feedback"]) else row["vect_growth_feedback"],
        }
        for _, row in df.iterrows()
    ]

	 for chunk in chunk_data(data, chunk_size):
	        try:
	            result = search_client.upload_documents(documents=chunk)
	            print(f"Uploaded {len(chunk)} documents successfully.")
	        except Exception as e:
	            print(f"An error occurred during document upload: {e}")
	            return None


upload_documents_to_search_client(df)

Some of the vector embedding colums have no lists or no values because there were no texts in the textual feedback columns, so because of that it would always say null values can't be uploaded to these vector fields. Can someone please help with this?

Azure AI Search
Azure AI Search
An Azure search service with built-in artificial intelligence capabilities that enrich information to help identify and explore relevant content at scale.
1,151 questions
0 comments No comments
{count} votes

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.