October 2024
These features and Azure Databricks platform improvements were released in October 2024.
Note
Releases are staged. Your Azure Databricks account might not be updated until a week or more after the initial release date.
Enable cost attribution of fine-grained access control on single user compute
October 31, 2024
This release includes a change to assist with the cost attribution of fine-grained access control on single user compute. With this change, when the single user compute passes a query to serverless compute to run data filtering, any custom tags assigned to the single user compute are propagated and recorded in the system.billing.usage table. See Fine-grained access control on single user compute.
Databricks Assistant helps optimize SQL queries
October 31, 2024
Improve the efficiency of SQL queries with syntax warnings and the /optimize slash command. Optimizations pop up in real time, helping you identify inefficiencies before execution.
See Use slash commands for prompts.
Databricks Runtime 15.1 series support ends
October 30, 2024
Support for Databricks Runtime 15.1 and Databricks Runtime 15.1 for Machine Learning ended on October 30. See Databricks support lifecycles.
Llama 2 70B Chat model retirement in Foundation Model APIs pay-per-token
October 30, 2024
Foundation Model APIs pay-per-token endpoints no longer support the Llama 2 70B Chat model. This model is now retired, see Retired models for a recommended replacement model.
Enable a metastore to be assigned automatically to new workspaces
October 28, 2024
Account admins can now set a Unity Catalog metastore to be assigned automatically when a new workspace is created in the same region, enabling Unity Catalog by default in those workspaces. See Enable a metastore to be automatically assigned to new workspaces.
Structured outputs supported on Mosaic AI Model Serving (Public Preview)
October 16, 2024
Structured outputs are now supported on Mosaic AI Model Serving as part of Foundation Model APIs. You can now specify a JSON schema to format responses generated from your chat models. See Structured outputs on Azure Databricks.
Fine-grained access control on single user compute is generally available
October 16, 2024
In Databricks Runtime 16.0 and above, fine-grained access control on single user compute is generally available. In workspaces enabled for serverless compute, if a query is run on supported compute such as single user compute and the query accesses any of the following objects, the compute resource passes the query to the serverless compute to run data filtering:
- Views defined over tables on which the user does not have the
SELECTprivilege. - Dynamic views.
- Tables with row filters or column masks applied.
- Materialized views and Streaming tables.
See Fine-grained access control on single user compute.
AI Functions now available in EU regions
October 15, 2024
AI Functions powered by Foundation Model APIs are now available in EU regions: northeurope and westeurope. See AI and machine learning.
Attribute serverless usage with budget policies (Public Preview)
October 15, 2024
To assist with serverless billing attribution, workspace admins can now create and assign budget policies to users, groups, or service principals. Budget policies enforce custom tags on all serverless usage incurred by the policy assignee. This allows for granular billing attribution of serverless usage in notebooks, jobs, and pipelines.
For more information, see Attribute serverless usage with budget policies.
Serverless compute automatically enabled on eligible accounts
October 15, 2024
Azure Databricks has started making serverless compute for notebooks, jobs, and Delta Live Tables available by default on eligible accounts, beginning with accounts on the Premium plan. See Connect to serverless compute.
Databricks Assistant Autocomplete is now generally available
October 10, 2024
Assistant Autocomplete provides intelligent inline code suggestions as you type to help you write code more quickly and efficiently. For more information, see Get inline code suggestions: Python and SQL examples and the announcement.
Filters on notebook results tables now persist
October 8, 2024
Notebooks results tables now persist filters when you reopen a notebook. See Filter results.
Improvements to comments in notebooks
October 8, 2024
Improvements to the UI for commenting in notebooks make comments easier to add and find. For more information, see Code comments.
Databricks Runtime 16.0 (Beta)
October 8, 2024
Databricks Runtime 16.0 and Databricks Runtime 16.0 ML are now available as Beta releases.
See Databricks Runtime 16.0 and Databricks Runtime 16.0 for Machine Learning.
Mosaic AI Model Serving now supports batch inference
October 8, 2024
Mosaic AI Model Serving now supports batch LLM inference using ai_query. See Perform batch inference using ai_query.
Create secure data and AI apps for your users with Databricks Apps (Public Preview)
October 8, 2024
With Databricks Apps, you can use popular Python frameworks to create applications that run in the Databricks platform and use the resources and features of the Databricks platform, including Unity Catalog for governance, Databricks SQL to query data, AI features such as model serving, and the already configured security rules in your workspaces. See What is Databricks Apps?.
More reliable reloading of modified Python modules with improvements to autoreload
October 8, 2024
In Databricks Runtime 16.0 and above, updates to the autoreload extension improve the safety and reliability of reloading modified Python modules imported from workspace files. With these changes, autoreload, when possible, reloads only the portion of a module that has changed instead of the entire module. Additionally, Databricks now automatically suggests using the autoreload extension if the module has changed since its last import. See Autoreload for Python modules.
Salesforce ingestion connector simplification
October 7, 2024
The Salesforce ingestion connector now automates source setup, which previously required manual configuration. To follow the simplified Salesforce ingestion process, see Ingest data from Salesforce.
Serverless compute now available in additional regions
October 2, 2024
Serverless compute for notebooks, workflows, and Delta Live Tables is now available in the following regions:
germanywestcentraluaenorthfrancecentralswedencentral
For more on serverless compute, see Connect to serverless compute.
These regions also support private connectivity for serverless compute. See Configure private connectivity from serverless compute.