New vector stores for Azure OpenAI Assistants fail after a few days

Question

I am experiencing a recurring issue with newer vector stores created for Azure OpenAI Assistants after December 2024. Assistants linked to these vector stores stop working correctly after a few days, causing significant delays in responses and failure to retrieve data from the vector store.

Key Observations

1. Previous Assistants: Assistants created before December 2024 using vector stores do not have this issue.
2. Model Impact: Changing models (e.g., GPT-4o or GPT-4o mini) does not resolve the problem.
3. Symptoms: No error messages appear, but:
   • Responses take a very long time.
     • File-based searches fail, and the assistant cannot provide answers based on the documents in the vector store.
       • In the Sandbox, the portal displays: "Söker efter referenser med ... Filer" (Searching for references with ... Files) during attempts to retrieve files.

Troubleshooting Steps Tried

• Removing and re-uploading files to the vector store: No improvement.
• Replacing files in the vector store: No improvement.
• Creating a new assistant linked to the same vector store: No improvement.
• Changing the GPT model: No improvement.
• Deleting and recreating the vector store with the exact same files: Temporary fix, but the issue recurs after a few days.

Business Impact

This issue significantly disrupts operations as customers relying on documents in the vector store are unable to get accurate responses. The failure to retrieve document-based answers creates a major bottleneck in productivity and usability of the assistant.

Details

I am experiencing a recurring issue with newer vector stores created for Azure OpenAI Assistants after December 2024. Assistants linked to these vector stores stop working correctly after a few days, causing significant delays in responses and failure to retrieve data from the vector store.

Key Observations

1. Previous Assistants: Assistants created before December 2024 using vector stores do not have this issue.
2. Model Impact: Changing models (e.g., GPT-4o or GPT-4o mini) does not resolve the problem.
3. Symptoms: No error messages appear, but:
   • Responses take a very long time.
     • File-based searches fail, and the assistant cannot provide answers based on the documents in the vector store.
       • In the Sandbox, the portal displays: "Söker efter referenser med ... Filer" (Searching for references with ... Files) during attempts to retrieve files.

Troubleshooting Steps Tried

• Removing and re-uploading files to the vector store: No improvement.
• Replacing files in the vector store: No improvement.
• Creating a new assistant linked to the same vector store: No improvement.
• Changing the GPT model: No improvement.
• Deleting and recreating the vector store with the exact same files: Temporary fix, but the issue recurs after a few days.

Business Impact

This issue significantly disrupts operations as customers relying on documents in the vector store are unable to get accurate responses. The failure to retrieve document-based answers creates a major bottleneck in productivity and usability of the assistant.

Answer 1

I wanted to update you regarding the issue we’ve been troubleshooting. Today, when testing one of the vector archives that had previously stopped working, it unexpectedly functioned as intended. The problem that was consistently reproducible before now no longer appears.

It’s possible that a backend adjustment has been made from Microsoft perspective, which might have resolved the issue. As the error is no longer present, it’s unfortunately impossible to continue troubleshooting further at this time.

Thank you @Manas Mohanty so much for your patience and all your efforts in helping investigate this matter. I will keep monitoring the situation.

Best regards, Benjamin