GPT-3.5 Turbo Deployment Issue: Cross-Region Integration Challenges with AI Search

Question

I have deployed the GPT-3.5 Turbo model, and my AI search (index and data source) is hosted in Sweden Central. The OpenAI service was created in Sweden, but the GPT-3.5 Turbo model is available only in France Central. Whenever I interact with the model, it responds with "document not found."

I specifically require the GPT-3.5 Turbo model because it serves as the completion model essential for integrating LLM agents. These LLM agents need to interact seamlessly with my database to process and retrieve information effectively. Due to this dependency, both the GPT-3.5 Turbo model and the LLM agents must function together properly. Switching to other models is not an option, as the completion capabilities of GPT-3.5 Turbo are critical for my use case.

Answer 1

Hello Pavankumar Purilla,

Welcome to the Microsoft Q&A and thank you for posting your questions here.

I understand that you are having issues with GPT-3.5 Turbo Deployment, having cross-Region Integration Challenges with AI Search.

The issue arises because the GPT-3.5 Turbo model is hosted in France Central, while the AI search service is hosted in Sweden Central. Azure services generally operate best when hosted in the same region due to reduced latency and optimized networking. The "document not found" error likely results from misconfigured cross-region communication or unsupported interaction across regions.

Please, kindly review this recommended solution:

1. Verify Cross-Region Access:
   • Ensure the AI search service in Sweden Central has the necessary permissions to access the GPT-3.5 Turbo model endpoint in France Central.
   • Use Azure's private endpoints or service endpoints to securely establish connectivity between Sweden Central and France Central. Check the network and firewall rules.
2. Test Latency:
   • Use Azure tools like Network Performance Monitor to test the latency between the two regions. Ensure it is within acceptable limits for your use case.
3. Migrate AI Search to France Central:
   • If latency or connectivity issues persist, consider migrating your AI search index and data source to France Central. This will ensure the services are co-located, eliminating latency or cross-region restrictions.
   • Alternative: Use Azure Synapse or Data Factory to replicate the data source in France Central for processing while keeping the original in Sweden Central.
4. Deploy a Custom API Layer:
   • Create an intermediate API in France Central that interacts with the GPT-3.5 Turbo model and proxies requests from Sweden Central. This minimizes direct cross-region communication.
5. Engage Azure Support:
   • If none of the above resolves the issue, escalate to Azure Support with detailed logs of the "document not found" error for further investigation.

I hope this is helpful! Do not hesitate to let me know if you have any other questions.

---

Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful.