what is difference between AKS and machine learning compute

Question

Azure offers managed compute resources, including Azure Kubernetes Service (AKS) or Azure Machine Learning Compute.

I want to upload fine-tuned LLM like Qwen2.5 or llama3.1 to Azure and online serving with these fine-tuned models, what is difference between AKS and Azure machine learning compute ?

Answer 1

Hi Michael Dong

Welcome to Microsoft Q&A Forum, thank you for posting your query here!

AKS and Azure Machine Learning Compute have different purposes when it comes to deploying and managing machine learning models. AKS is a general-purpose container orchestration platform, while Azure Machine Learning Compute is specifically designed for machine learning tasks. It offers specialized features and optimizations for model training and deployment, making it a more tailored solution for machine learning workloads.

To serve your fine-tuned Llama 3.1 model and Qwen 2.5-1.5B model in Azure using managed compute. can you please refer: fine tune

You can attached AKS cluster through attach compute option and deploy LLM like llama or phi 3 through KAITO add on in AKS.  

Kindly refer the below documentation: https://learn.microsoft.com/en-us/azure/machine-learning/how-to-attach-kubernetes-anywhere?view=azureml-api-2

https://learn.microsoft.com/en-us/azure/aks/ai-toolchain-operator#enable-the-ai-toolchain-operator-add-on-on-an-aks-cluster

Hope this helps. Do let us know if you any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Thank You.