Κοινή χρήση μέσω

DLT release notes and the release upgrade process

  • Άρθρο

This article explains the DLT release process, how the DLT runtime is managed, and provides links to release notes for each DLT release.

DLT runtime channels

Note

To see the Databricks Runtime versions used with a DLT release, see the release notes for that release.

DLT clusters use runtimes based on Databricks Runtime release notes versions and compatibility. Databricks automatically upgrades the DLT runtimes to support enhancements and upgrades to the platform. You can use the channel field in the DLT pipeline settings to control the DLT runtime version that runs your pipeline. The supported values are:

  • current to use the current runtime version.
  • preview to test your pipeline with upcoming changes to the runtime version.

By default, your pipelines run using the current runtime version. Databricks recommends using the current runtime for production workloads. To learn how to use the preview setting to test your pipelines with the next runtime version, see Automate testing of your pipelines with the next runtime version.

Important

Features marked as generally available or Public Preview are available in the current channel.

For more information about DLT channels, see the channel field in the DLT pipeline settings.

To understand how DLT manages the upgrade process for each release, see How do DLT upgrades work?.

How do I find the Databricks Runtime version for a pipeline update?

You can query the DLT event log to find the Databricks Runtime version for a pipeline update. See Runtime information.

DLT release notes

DLT release notes are organized by year and week-of-year. Because DLT is versionless, both workspace and runtime changes take place automatically. The following release notes provide an overview of changes and bug fixes in each release:

How do DLT upgrades work?

DLT is considered to be a versionless product, which means that Databricks automatically upgrades the DLT runtime to support enhancements and upgrades to the platform. Databricks recommends limiting external dependencies for DLT pipelines.

Databricks proactively works to prevent automatic upgrades from introducing errors or issues to production DLT pipelines. See DLT upgrade process.

Especially for users that deploy DLT pipelines with external dependencies, Databricks recommends proactively testing pipelines with preview channels. See Automate testing of your pipelines with the next runtime version.

DLT upgrade process

Databricks manages the Databricks Runtime used by DLT compute resources. DLT automatically upgrades the runtime in your Azure Databricks workspaces and monitors the health of your pipelines after the upgrade.

If DLT detects that a pipeline cannot start because of an upgrade, the runtime version for the pipeline reverts to the previous version that is known to be stable, and the following steps are triggered automatically:

  • The pipeline’s DLT runtime is pinned to the previous known-good version.
  • Databricks support is notified of the issue.
    • If the issue is related to a regression in the runtime, Databricks resolves the issue.
    • If the issue is caused by a custom library or package used by the pipeline, Databricks contacts you to resolve the issue.
  • When the issue is resolved, Databricks initiates the upgrade again.

Important

DLT only reverts pipelines running in production mode with the channel set to current.

Automate testing of your pipelines with the next runtime version

To ensure changes in the next DLT runtime version do not impact your pipelines, use the DLT channels feature:

  1. Create a staging pipeline and set the channel to preview.
  2. In the DLT UI, create a schedule to run the pipeline weekly and enable alerts to receive an email notification for pipeline failures. Databricks recommends scheduling weekly test runs of pipelines, especially if you use custom pipeline dependencies.
  3. If you receive a notification of a failure and are unable to resolve it, open a support ticket with Databricks.

Pipeline dependencies

DLT supports external dependencies in your pipelines; for example, you can install any Python package using the %pip install command. DLT also supports using global and cluster-scoped init scripts. However, these external dependencies, particularly init scripts, increase the risk of issues with runtime upgrades. To mitigate these risks, minimize using init scripts in your pipelines. If your processing requires init scripts, automate testing of your pipeline to detect problems early; see Automate testing of your pipelines with the next runtime version. If you use init scripts, Databricks recommends increasing your testing frequency.