Azure development issue

Mumuksha Paliwal 0

issues are regarding the Connection(in Azure function app) using the GRPC port in Weaviate DB Instance, after we have setup the SSL cert. for the Instance

Ryan Hill 29,386 Reputation points Microsoft Employee

2024-11-26T19:33:59.04+00:00

Hi @Mumuksha Paliwal, in order for the community to best assist you, can add some additional details? What issue are you having? Can you elaborate on how Weaviate Db instance is configured/hosted? Can you share any error message(s) you're seeing?
Mumuksha Paliwal 0 Reputation points

2024-11-27T10:22:24.2933333+00:00
Hi @Ryan Hill

Here it is:

For our weaviate database initially we have 8vCPU and 8GB memory, but as we started inserting data into weaviate around 6.5M+ object we are started seeing instance crash if we perform multiple queries at a time and our memory usage is around 95-98% so we have decided to upgrade memory to 32GB and currently memory usage is ~58%. So can you please help us what type of instance configuration will be good to store 10m+ records (considering future) and high read and write operation and if there is any process we can follow to optimize the weaviate memory consumption.

We are using azure function (Flex Consumption plan) for our ETL pipeline deployment, there we are seeing drop in performance of data updation to weaviate. As we are using same script on our Azure VM the execution time is ~15 -20 sec for 1.5k records. But on Azure function application it is taking more then 30 mins to execute and ultimately function is getting timeout.Will upgrading our function app to App Servies plan can help us fix this issue.

Share via

Azure development issue

Your answer