What's new and planned for OneLake in Microsoft Fabric
Important
The release plans describe functionality that may or may not have been released yet. The delivery timelines and projected functionality may change or may not ship. Refer to Microsoft policy for more information.
OneLake is a single, unified, logical data lake for your whole organization. Like OneDrive, OneLake comes automatically with every Microsoft Fabric tenant and is designed to be the single place for all your analytics data.
Any data in OneLake works with out-of-the-box governance such as data lineage, data protection, certification, catalog integration, etc. and is ultimately under the control of a tenant admin. Within a tenant, workspaces enable different parts of the organization to work independently while still contributing to the same data lake.
OneLake is open at every level. OneLake supports the same ADLS Gen2 APIs and SDKs to be compatible with existing ADLS Gen2 applications and can support any type of file, structured or unstructured.
OneLake aims to give you the most value possible out of a single copy of data. With OneLake shortcuts, you can unify your data across domains, clouds, and accounts by creating a reference to data stored in other file locations such as other OneLake locations or ADLS or S3 without data movement or duplication. You can also use the same data across multiple analytical engines because Fabric engines store all tabular data in the open parquet formats.There's no longer a need to copy data just to use it with another engine.
To learn more, see the documentation.
Investment areas
| Feature | Estimated release timeline |
|---|---|
| OneLake table shortcuts to Iceberg data | Q4 2024 |
| OneLake security model | Q1 2025 |
| OneLake data access roles general availability | Q1 2025 |
| Cache support for on-premises data gateway in Onelake shortcuts | Q1 2025 |
| CI/CD support for Onelake shortcuts | Q1 2025 |
| Easy management of orphaned Onelake shortcuts | Q1 2025 |
| OneLake Encryption with Customer Managed Keys | Q2 2025 |
| Fabric OneLake catalog | Shipped (Q4 2024) |
| Databricks Unity Catalog support for OneLake | Shipped (Q3 2024) |
| OneLake shortcuts to on-premises data | Shipped (Q2 2024) |
| Shortcuts Google cloud storage | Shipped (Q1 2024) |
| Shortcuts API | Shipped (Q1 2024) |
| Smart caching for Amazon S3 shortcuts | Shipped (Q4 2023) |
OneLake table shortcuts to Iceberg data
Estimated release timeline: Q4 2024
Release Type: Public preview
With Microsoft OneLake, you will be able to create table shortcuts to Iceberg data in other locations, and this data will appear automatically with the Delta Lake format.
OneLake automatically produces virtualized Delta Lake metadata and makes it available through the OneLake shortcut. This means you can use your Iceberg data with all Fabric workloads that work with Delta Lake formatted data.
OneLake security model
Estimated release timeline: Q1 2025
Release Type: Public preview
Managing data security across multiple analytical engines and copies of data is challenging. OneLake and Fabric simplify this by enabling the use of a single data copy across multiple analytical engines without any data movement or duplication. Taking the "one copy" concept further, OneLake is also enhancing security with a finer-grain model, allowing for table and folder access in addition to row and column level security. These security definitions live with the data and travel across shortcuts to wherever the data is used. Security defined at OneLake is consistantly enforced across multiple Fabric analytical engines.
OneLake data access roles general availability
Estimated release timeline: Q1 2025
Release Type: General availability
OneLake data access roles build upon the existing capabilities of OneLake’s security model to increase the granularity at which security can be applied within a Fabric data item. This feature adds an inheritable RBAC (role-based access control) model that simplifies user and permissions management for data in OneLake. You can define security roles that grant read access to specific folders in OneLake, and assign them to users or groups. The access permissions determine what folders users see when accessing the lake view of the data, either through the lakehouse UX, notebooks, or OneLake APIs.
Cache support for on-premises data gateway in Onelake shortcuts
Estimated release timeline: Q1 2025
Release Type: General availability
Shortcut caching can be used to reduce egress costs associated with cross-cloud data access. As files are read through an external shortcut, the files are stored in a cache for the Fabric workspace. Subsequent read requests are served from cache rather than the remote storage provider. With the introduction of cache support for the On-premises-data gateway, you can leverage these capabilities while connecting to on-prem data or data behind a firewall.
CI/CD support for Onelake shortcuts
Estimated release timeline: Q1 2025
Release Type: Public preview
Microsoft Fabric's lifecycle management tools provide a standardized system for communication and collaboration between all members of the development team throughout the life of the product. With the introduction of CI/CD support for Onelake shortcuts, you can easiliy manage your deploy,ent of Onelake shortcuts across environments.
Easy management of orphaned Onelake shortcuts
Estimated release timeline: Q1 2025
When people switch teams, leave the organization or lose certain access, your Onelkae shortcuts can become unusable. With the new management capabilities of these orphaned shortcuts, you can easly restore your shortcuts with no business impact.
OneLake Encryption with Customer Managed Keys
Estimated release timeline: Q2 2025
Release Type: Public preview
As part of our strategic roadmap to enhance data security and compliance, we are planning to introduce support for Customer-Managed Keys (CMK) in OneLake. This will provide organizations with greater control over their data encryption keys. With CMK, users will be able to manage their own encryption keys, ensuring that their data is protected in accordance with their specific security policies and regulatory requirements. If the customer's key is revoked, Microsoft will lose access to customer data in OneLake.
Shipped feature(s)
Fabric OneLake catalog
Shipped (Q4 2024)
Release Type: General availability
OneLake catalog is a complete solution to explore, manage and govern your entire Fabric data estate. The OneLake catalog is an evolution of the OneLake data hub experience with enhancements that will help all Fabric users discover and manage trusted data, as well as provide governance for data owners with valuable insights, recommended actions, and tooling. The OneLake catalog comes with two tabs: Explore and Govern. In the Explore tab, users can explore and manage all the Fabric items they have access to in one central location with filters for domains, item type, owner, endorsement, tags, and more. Users can double click on a Fabric item to take actions or explore description, data lineage, permissions, and activity. And in the Govern tab, data owners can see an overview of their data with insights on endorsements, data labeling, data type, and more, and get recommended actions to enhance their data for better quality and compliance. The Explore tab is now generally available and the Govern tab will be coming soon in preview.
Databricks Unity Catalog support for OneLake
Shipped (Q3 2024)
Release Type: Public preview
Azure Databricks Unity Catalog Integration with Microsoft Fabric
You will be able to access Azure Databricks Unity Catalog tables directly in Microsoft Fabric, making it even easier to unify Azure Databricks with Microsoft Fabric. From the Fabric portal, you can create and configure a new Azure Databricks Unity Catalog item in Fabric with just a few clicks. You can add a full catalog, a schema, or even individual tables to link and the management of this Azure Databricks item in OneLake—a shortcut connected to Unity Catalog—is automatically taken care of for you. This data acts like any other data in OneLake—you can write SQL queries or use it with any other workloads in Fabric including Power BI via Direct Lake mode. When the data is modified or tables are added, removed, or renamed in Azure Databricks, the data in Fabric will remain always in sync. This new integration makes it simple to unify Azure Databricks data in Fabric and seamlessly use it across every Fabric workload.
OneLake shortcuts to on-premises data
Shipped (Q2 2024)
Release Type: Public preview
Microsoft OneLake shortcuts is expanding to include on-premises and network-restricted data sources. With this capability, you can unify your on-prem and cloud data in OneLake.
During creation of shortcuts to AWS S3, Google Cloud Storage, or S3 compatible buckets, you will be able to optionally select an on-premises data gateway (OPDG) to establish connectivity.
Shortcuts Google cloud storage
Shipped (Q1 2024)
Release Type: Public preview
OneLake will expand shortcut support to Google Cloud Storage, allowing virtualization of data without moving or duplicating it. This enables the integration of Google Cloud Storage data with Microsoft services like ADLS gen2, OneLake, Dataverse, and Amazon S3. The data will appear and work as if it was in OneLake and gives you a simple data lake that can span clouds.
Shortcuts API
Shipped (Q1 2024)
Release Type: Public preview
A public REST API to automate creation and management of shortcut scenarios.
Smart caching for Amazon S3 shortcuts
Shipped (Q4 2023)
Release Type: Public preview
Smart caching for Amazon S3 shortcuts reduces egress costs and enhances performance by bringing data closer to the compute engine. Smart caching will egress data from S3 once, cache it locally in OneLake for a certain period. This way it eliminates the need for repeated data retrieval from S3. The cached data can be reused across multiple users, analytical engines, and scenarios that optimize the value a single egress.