Mapping data flow transformation overview

APPLIES TO: Azure Data Factory Azure Synapse Analytics

Tip

Try out Data Factory in Microsoft Fabric, an all-in-one analytics solution for enterprises. Microsoft Fabric covers everything from data movement to data science, real-time analytics, business intelligence, and reporting. Learn how to start a new trial for free!

Data flows are available both in Azure Data Factory and Azure Synapse Pipelines. This article applies to mapping data flows. If you are new to transformations, please refer to the introductory article Transform data using a mapping data flow.

Below is a list of the transformations currently supported in mapping data flow. Click on each transformations to learn its configuration details.

Name	Category	Description
Aggregate	Schema modifier	Define different types of aggregations such as SUM, MIN, MAX, and COUNT grouped by existing or computed columns.
Alter row	Row modifier	Set insert, delete, update, and upsert policies on rows.
Assert	Row modifier	Set assert rules for each row.
Cast	Schema modifier	Change column data types with type checking.
Conditional split	Multiple inputs/outputs	Route rows of data to different streams based on matching conditions.
Derived column	Schema modifier	Generate new columns or modify existing fields using the data flow expression language.
External call	Schema modifier	Call external endpoints inline row-by-row.
Exists	Multiple inputs/outputs	Check whether your data exists in another source or stream.
Filter	Row modifier	Filter a row based upon a condition.
Flatten	Formatters	Take array values inside hierarchical structures such as JSON and unroll them into individual rows.
Flowlet	Flowlets	Build and include custom re-usable transformation logic.
Join	Multiple inputs/outputs	Combine data from two sources or streams.
Lookup	Multiple inputs/outputs	Reference data from another source.
New branch	Multiple inputs/outputs	Apply multiple sets of operations and transformations against the same data stream.
Parse	Formatters	Parse text columns in your data stream that are strings of JSON, delimited text, or XML formatted text.
Pivot	Schema modifier	An aggregation where one or more grouping columns has its distinct row values transformed into individual columns.
Rank	Schema modifier	Generate an ordered ranking based upon sort conditions
Select	Schema modifier	Alias columns and stream names, and drop or reorder columns
Sink	-	A final destination for your data
Sort	Row modifier	Sort incoming rows on the current data stream
Source	-	A data source for the data flow
Stringify	Formatters	Turn complex types into plain strings
Surrogate key	Schema modifier	Add an incrementing non-business arbitrary key value
Union	Multiple inputs/outputs	Combine multiple data streams vertically
Unpivot	Schema modifier	Pivot columns into row values
Window	Schema modifier	Define window-based aggregations of columns in your data streams.