Inference as a service (AWS Bedrock equivalent)

Question

Hey guys,

I am trying to see if I can do inference as a service like AWS Bedrock. One problem with Bedrock is that it doesn't run Lora based models, but if I can do this over Azure AI it would be great.

Thanks a lot!

Peter

Answer 1

Hi Peter Ma,
Greetings & Welcome to the Microsoft Q&A forum! Thank you for sharing your query.

Azure AI offers a robust solution for inference as a service, similar to AWS Bedrock. You can use the Azure AI Model Inference API, which allows you to consume various models, including those from OpenAI, Cohere, and Meta, using a unified API.

Regarding LoRA-based models, Azure AI does support them. You can fine-tune and deploy LoRA fine-tuned models, such as Llama 3.1 8B.
For more information: https://techcommunity.microsoft.com/blog/aiplatformblog/the-future-of-ai-deploying-your-lora-fine-tuned-llama-3-1-8b-on-azure-ai-why-its/4276562

I hope this information helps.