Can I use Azure Blob files to create a vector store

Question

Hello,

From this page I was able to set up a Azure Blob stroage and conenct to the an open ai chat assistant - https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/use-your-data?tabs=blob-storage%2Ccopilot

However, I am interested in creating vector stores to use for file search in the Assistants Playground (preview). Is this supported?

Basically, what I want to do is to create a vector store based off some data uploaded to azure with some set expiry. When the vector expires, I also want to delete the files uploaded to azure (keep the # of files uploaded low, clean up when possible, and internal data privacy/security reasons).

I was looking into Azure Blob Storage lifecycle management policies to help align when files should be deleted. However, it seems that to create a vector store for AI Assistant, I must upload it to a different storage in Azure for it to be converted into a vector store. Is that correct?

Any suggestions or insights are deeply appreciated

Thanks!

Answer 1

Hi Wilson C,

Thanks for your follow-up!

I see the distinction you're making between Azure AI Search vector stores and the vector store concept used in the AI Assistant file search feature.

The vector store in AI Search is primarily used for semantic search and retrieval-augmented generation (RAG) scenarios, where documents are indexed with embeddings for similarity-based searches. However, the vector store for file search in the Assistants Playground (preview) works differently and is specific to Azure OpenAI's Assistants API.

Key Differences:

AI Search Vector Store:

• Used for semantic search over large datasets.
• Requires indexing data in Azure AI Search, where embeddings are created and stored.
• Works with vector search queries to retrieve relevant results.

File Search Vector Store (Assistants API):

• Used for retrieving files uploaded to an Assistant session.
• Requires files to be uploaded and processed into embeddings within Azure OpenAI Assistants.
• The vector store is created within Azure OpenAI Assistants, and it is separate from Azure AI Search.

What This Means for Your Use Case:

• If you're using Assistants Playground (preview) file search, your files need to be uploaded directly to the Assistant’s vector store rather than an external Azure AI Search index.
• This means lifecycle management via Azure Blob Storage policies wouldn’t directly apply to the Assistants vector store, as file retention there is managed differently.
• You may need to handle manual deletion or use Azure automation to clean up both Blob Storage files and Assistant-uploaded vectors after a set expiry.

I hope this is helpful! Do not hesitate to let me know if you have any other questions.

Thanks