Introduction
Microsoft Fabric offers a unified solution for data engineering, integration, and analytics. A crucial step in end-to-end analytics is data ingestion. Dataflows Gen2 are used to ingest and transform data from multiple sources, and then land the cleansed data to another destination. They can be incorporated into data pipelines for more complex activity orchestration, and also used as a data source in Power BI.
Imagine you work for a retail company with stores across the globe. As a data engineer, you need to prepare and transform data from various sources into a format that is suitable for data analysis and reporting. The business requests a semantic model that consolidates disparate data sources from the different stores. Dataflows Gen2 allow you to prepare the data to ensure consistency, and then stage the data in the preferred destination. They also enable reuse and make it easy to update the data. Without a dataflow, you'd have to manually extract and transform the data from every source, which is time-consuming and prone to errors.
In this module, we explain how to use Dataflows Gen2 in Microsoft Fabric to meet your data ingestion needs.