Getting from Azure Data Factory to Data Factory in Microsoft Fabric
Data Factory in Microsoft Fabric is the next generation of Azure Data Factory which provides cloud-scale data movement and data transformation services that allow you to solve the most complex ETL scenarios. It's intended to make your experience easy to use, powerful, and truly enterprise-grade. This article compares the differences between Azure Data Factory and Data Factory in Microsoft Fabric.
Feature mapping
In the modern experience of Data Factory in Fabric, there are some different features concepts compared to Azure Data Factory. Detail features mapping is presented as the table below.
| Azure Data Factory | Data Factory in Fabric | Description |
|---|---|---|
| Pipeline | Data pipeline | Data pipeline in Fabric is better integrated with the unified data platform including Lakehouse, Datawarehouse, and more. |
| Mapping dataflow | Dataflow Gen2 | Dataflow Gen2 provides easier experience to build transformation. We are in progress of letting more functions of mapping dataflow supported in Dataflow Gen2 |
| Activities | Activities | We are in progress to make more activities of ADF supported in Data Factory in Fabric. Data Factory in Fabric also has some newly attracted activities like Office 365 Outlook activity. Details are in Activity overview. |
| Dataset | Not Applicable | Data Factory in Fabric doesn’t have dataset concepts. Connection will be used for connecting each data source and pull data. |
| Linked Service | Connections | Connections have similar functionality as linked service, but connections in Fabric have more intuitive way to create. |
| Triggers | Schedule triggers and file event triggers | Fabric can use the scheduler and Reflex events to automatically run pipelines. File event triggers are supported natively in pipelines in Microsoft Fabric Data Factory. |
| Publish | Save, Run | For pipeline in Fabric, you don’t need to publish to save the content. Instead, you can use Save button to save the content directly. When you click Run button, it will save the content before running pipeline. |
| Autoresolve and Azure Integration runtime | Not Applicable | In Fabric, we don’t have the concept of Integration runtime. |
| Self-hosted integration runtimes | On-premises Data Gateway | On-premises Data Gateway enables access to on-premises data via the Fabric Data Factory. Details are in How to access on-premises data sources in Data Factory for Microsoft Fabric. |
| Azure-SSIS integration runtimes | To be determined | The capability in Fabric hasn’t confirmed the roadmap and design. |
| MVNet and Private End Point | To be determined | The capability in Fabric hasn’t confirmed the roadmap and design. |
| Expression language | Expression language | Expression language is similar in ADF and Fabric. |
| Authentication type in linked service | Authentication kind in connection | Authentication kind in Fabric pipeline already supported popular authentication types in ADF, and more authentication kinds will be added. |
| CI/CD | CI/CD | CI/CD capability in Fabric Data Factory will be coming soon. |
| Export and Import ARM | Save as | Save as is available in Fabric pipeline to duplicate a pipeline. |
| Monitoring | Monitoring, Run history | The monitoring hub in Fabric has more advanced functions and modern experience like monitoring across different workspaces for better insights. |
Data pipeline of data factory in Microsoft Fabric
There are many exciting features of data pipeline in Data Factory of Microsoft Fabric. Leveraging these features, you can feel the power of pipeline in Fabric.
Lakehouse/Datawarehouse integration
Lakehouse and Data Warehouse are available as source and destination in Pipeline of Fabric, so it’s extremely convenient for you to build your own projects integrated with Lakehouse and Datawarehouse.
Office 365 outlook activity
Office 365 outlook activity provides an intuitive and simple way to send customized email notification about info of pipeline and activity, and output of pipeline by easy configuration.
Get data experience
A modern and easy Get Data experience is provided in Data Factory in Fabric, so it’s super-fast for you to set up your copy pipeline and create a new connection.
Modern monitoring experience
With the combined capabilities of the monitoring hub and the items of Data Factory, such as data flows and data pipelines, we can get a full view of all the workloads and drill into any activity within a data factory experience. It’s also convenient for you to do the cross-workspace analysis through monitoring hub.
The pipeline copy monitoring results provide breakdown detail of the Copy activity. By selecting the run details button (with the glasses icon highlighted) to view the run details. Expand the Duration breakdown, you can know the time duration of each stage in copy activity.
Save as
Save as in Fabric pipeline provides a convenient way for you to duplicate an existing pipeline for other development purposes.
