Set the target catalog and schema
The Destination section of the pipeline configuration UI sets the default catalog and schema for a pipeline. This default catalog and schema are used for all dataset definitions and table reads.
Note
Legacy publishing mode uses the LIVE
virtual schema to achieve similar behavior. In the default publishing mode (used by all new pipelines), the LIVE
keyword is ignored. See LIVE schema (legacy).
Target a dataset in a different catalog or schema
Delta Live Tables supports three-tier identifier resolution semantics. Databricks recommends using fully-qualified identifiers for queries and statements that target datasets other than the defaults configured for your pipeline. See Unity Catalog identifier resolution.
Delta Live Tables source code supports the SQL commands USE CATALOG catalog_name
and USE SCHEMA schema_name
. Run these commands to set the current catalog and schema scoped to the file or notebook containing these commands. Operations that follow these commands in the source code file and use unqualified or partially-qualified identifiers resolve to the current catalog and schema rather than the default values set in the pipeline configuration. See What is the current catalog and schema?.
What happens if a dataset does not exist?
The following table describes the behavior when Delta Live Tables source code references datasets that don’t exist:
Operation | Outcome |
---|---|
Read | If a table, materialized view, streaming table, or view does not exist for the specified identifier, the update fails. |
Write | If a materialized view, streaming table, view, or sink does not exist for the specified identifier, the update attempts to create the dataset. If necessary, the update also creates the specified schema. |
Important
You might receive an error message that states a dataset does not exist if you do not have sufficient privileges to view the dataset.
You must have sufficient privileges to read, write, and create datasets with Delta Live Tables. See Requirements.