Open mirroring in Microsoft Fabric (Preview)
Mirroring in Fabric provides an easy experience to avoid complex ETL (Extract Transform Load) and integrate your existing data into OneLake with the rest of your data in Microsoft Fabric. You can continuously replicate your existing data directly into Fabric's OneLake. Inside Fabric, you can unlock powerful business intelligence, artificial intelligence, Data Engineering, Data Science, and data sharing scenarios.
Open mirroring enables any application to write change data directly into a mirrored database in Fabric. Open mirroring is designed to be extensible, customizable, and open. It's a powerful feature that extends mirroring in Fabric based on open Delta Lake table format.
Once the data lands in OneLake in Fabric, open mirroring simplifies the handling of complex data changes, ensuring that all mirrored data is continuously up-to-date and ready for analysis.
Important
This feature is in preview.
For a tutorial on configuring your open mirrored database in Fabric, see Tutorial: Configure Microsoft Fabric open mirrored databases.
Why use open mirroring in Fabric?
Open mirroring extends the Mirroring in Fabric capability to your own applications, or existing data providers to land data into a mirrored database within OneLake in Fabric. Once the data lands in the landing zone, the mirroring replication engine manages the complexity of changes and converts data into Delta Parquet, an analytics-ready format. In the OneLake, your data can be analyzed and consumed by all the experiences in Fabric.
Open mirroring meets your data replication needs if you:
- Use your own application to write data into the open mirroring landing zone per the open mirroring landing zone requirements and formats.
- Use one of our existing open mirroring partners to help you ingest data.
What analytics experiences are built in?
All types of mirrored databases are an item in Fabric Data Warehousing distinct from the Warehouse and SQL analytics endpoint.
Mirroring creates three items in your Fabric workspace:
- The mirrored database item. Mirroring manages the replication of data into OneLake and conversion into Delta Parquet format, and manages the complexity of the changes, in an analytics-ready format. This enables downstream scenarios like data engineering, data science, and more.
- A SQL analytics endpoint
- A default semantic model
Each open mirrored database has an autogenerated SQL analytics endpoint that provides a rich analytical experience on top of the Delta Tables created by the mirroring process. Users have access to familiar T-SQL commands that can define and query data objects but not manipulate the data from the SQL analytics endpoint, as it's a read-only copy. You can perform the following actions in the SQL analytics endpoint:
- Explore the tables that reference data in your Delta Lake tables.
- Create no code queries and views and explore data visually without writing a line of code.
- Develop SQL views, inline TVFs (Table-valued Functions), and stored procedures to encapsulate your semantics and business logic in T-SQL.
- Manage permissions on the objects.
- Query data in other Warehouses and Lakehouses in the same workspace.
In addition to the SQL query editor, there's a broad ecosystem of tooling that can query the SQL analytics endpoint, including SQL Server Management Studio (SSMS), the mssql extension with Visual Studio Code, and even GitHub Copilot.
Open mirroring cost considerations
Same as for all types of mirroring in Fabric, open mirroring offers a free terabyte of mirroring storage for every capacity unit (CU) you have purchased and provisioned. For example, if you purchase F64, you get 64 free terabytes worth of storage for your mirrored replicas. OneLake storage is billed only when the free mirroring storage limit is exceeded, or the capacity is paused.
In addition, the compute needed to manage the complexity of change data is free and it doesn't consume capacity. Requests to OneLake as part of the mirroring process consume capacity like normal with OneLake compute consumption.