Identify Azure Databricks workloads

  • 3 minutes

Azure Databricks offers capabilities for various workloads including Machine Learning and Large Language Models (LLM), Data Science, Data Engineering, BI and Data Warehousing, and Streaming Processing.

Data Science and Engineering

This workload is designed for data scientists and engineers who need to collaborate on complex data processing tasks. It provides an integrated environment with Apache Spark for big data processing in a data lakehouse, and supports multiple languages including Python, R, Scala, and SQL. The platform facilitates data exploration, visualization, and the development of data pipelines.

Diagram of Databricks data ingestion & data sources screen.

Machine Learning

The Machine Learning workload on Azure Databricks is optimized for building, training, and deploying machine learning models at scale. It includes MLflow, an open-source platform to manage the ML lifecycle, including experimentation, reproducibility, and deployment. It also supports various ML frameworks such as TensorFlow, PyTorch, and Scikit-learn, making it versatile for different ML tasks.

Diagram of Databricks Machine Learning screen.

SQL

The SQL workload is geared towards data analysts who primarily interact with data through SQL. It provides a familiar SQL editor, dashboards, and automatic visualization tools to analyze and visualize data directly within Azure Databricks. This workload is ideal for running quick ad-hoc queries and creating reports from large datasets.

Diagram of DatabricksSQL Editor screen.