Dataflow - Create Or Update
Create a DataflowResource
PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.IoTOperations/instances/{instanceName}/dataflowProfiles/{dataflowProfileName}/dataflows/{dataflowName}?api-version=2024-11-01URI Parameters
| Name | In | Required | Type | Description |
|---|---|---|---|---|
|
dataflow
|
path | True |
string |
Name of Instance dataflowProfile dataflow resource Regex pattern: |
|
dataflow
|
path | True |
string |
Name of Instance dataflowProfile resource Regex pattern: |
|
instance
|
path | True |
string |
Name of instance. Regex pattern: |
|
resource
|
path | True |
string |
The name of the resource group. The name is case insensitive. |
|
subscription
|
path | True |
string uuid |
The ID of the target subscription. The value must be an UUID. |
|
api-version
|
query | True |
string |
The API version to use for this operation. |
Request Body
| Name | Required | Type | Description |
|---|---|---|---|
| extendedLocation | True |
Edge location of the resource. |
|
| properties |
The resource-specific properties for this resource. |
Responses
| Name | Type | Description |
|---|---|---|
| 200 OK |
Resource 'DataflowResource' update operation succeeded |
|
| 201 Created |
Resource 'DataflowResource' create operation succeeded Headers
|
|
| Other Status Codes |
An unexpected error response. |
Security
azure_auth
Azure Active Directory OAuth2 Flow.
Type:
oauth2
Flow:
implicit
Authorization URL:
https://login.microsoftonline.com/common/oauth2/authorize
Scopes
| Name | Description |
|---|---|
| user_impersonation | impersonate your user account |
Examples
Dataflow_CreateOrUpdate
Sample request
PUT https://management.azure.com/subscriptions/F8C729F9-DF9C-4743-848F-96EE433D8E53/resourceGroups/rgiotoperations/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123?api-version=2024-11-01
{
"properties": {
"mode": "Enabled",
"operations": [
{
"operationType": "Source",
"name": "knnafvkwoeakm",
"sourceSettings": {
"endpointRef": "iixotodhvhkkfcfyrkoveslqig",
"assetRef": "zayyykwmckaocywdkohmu",
"serializationFormat": "Json",
"schemaRef": "pknmdzqll",
"dataSources": [
"chkkpymxhp"
]
},
"builtInTransformationSettings": {
"serializationFormat": "Delta",
"schemaRef": "mcdc",
"datasets": [
{
"key": "qsfqcgxaxnhfumrsdsokwyv",
"description": "Lorem ipsum odor amet, consectetuer adipiscing elit.",
"schemaRef": "n",
"inputs": [
"mosffpsslifkq"
],
"expression": "aatbwomvflemsxialv"
}
],
"filter": [
{
"type": "Filter",
"description": "Lorem ipsum odor amet, consectetuer adipiscing elit.",
"inputs": [
"sxmjkbntgb"
],
"expression": "n"
}
],
"map": [
{
"type": "NewProperties",
"description": "Lorem ipsum odor amet, consectetuer adipiscing elit.",
"inputs": [
"xsbxuk"
],
"expression": "txoiltogsarwkzalsphvlmt",
"output": "nvgtmkfl"
}
]
},
"destinationSettings": {
"endpointRef": "kybkchnzimerguekuvqlqiqdvvrt",
"dataDestination": "cbrh"
}
}
]
},
"extendedLocation": {
"name": "qmbrfwcpwwhggszhrdjv",
"type": "CustomLocation"
}
}
Sample response
{
"properties": {
"mode": "Enabled",
"operations": [
{
"operationType": "Source",
"name": "knnafvkwoeakm",
"sourceSettings": {
"endpointRef": "iixotodhvhkkfcfyrkoveslqig",
"assetRef": "zayyykwmckaocywdkohmu",
"serializationFormat": "Json",
"schemaRef": "pknmdzqll",
"dataSources": [
"chkkpymxhp"
]
},
"builtInTransformationSettings": {
"serializationFormat": "Delta",
"schemaRef": "mcdc",
"datasets": [
{
"key": "qsfqcgxaxnhfumrsdsokwyv",
"description": "Lorem ipsum odor amet, consectetuer adipiscing elit.",
"schemaRef": "n",
"inputs": [
"mosffpsslifkq"
],
"expression": "aatbwomvflemsxialv"
}
],
"filter": [
{
"type": "Filter",
"description": "Lorem ipsum odor amet, consectetuer adipiscing elit.",
"inputs": [
"sxmjkbntgb"
],
"expression": "n"
}
],
"map": [
{
"type": "NewProperties",
"description": "Lorem ipsum odor amet, consectetuer adipiscing elit.",
"inputs": [
"xsbxuk"
],
"expression": "txoiltogsarwkzalsphvlmt",
"output": "nvgtmkfl"
}
]
},
"destinationSettings": {
"endpointRef": "kybkchnzimerguekuvqlqiqdvvrt",
"dataDestination": "cbrh"
}
}
],
"provisioningState": "Succeeded"
},
"extendedLocation": {
"name": "qmbrfwcpwwhggszhrdjv",
"type": "CustomLocation"
},
"id": "/subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/resourceGroup123/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123",
"name": "jxhcpwgfkxqasbexkookvxk",
"type": "zkuozvgjseokfchkscoswthzjdry",
"systemData": {
"createdBy": "ssvaslsmudloholronopqyxjcu",
"createdByType": "User",
"createdAt": "2024-08-09T18:13:29.389Z",
"lastModifiedBy": "gnicpuszwd",
"lastModifiedByType": "User",
"lastModifiedAt": "2024-08-09T18:13:29.389Z"
}
}Azure-AsyncOperation: https://contoso.com/operationstatus{
"properties": {
"mode": "Enabled",
"operations": [
{
"operationType": "Source",
"name": "knnafvkwoeakm",
"sourceSettings": {
"endpointRef": "iixotodhvhkkfcfyrkoveslqig",
"assetRef": "zayyykwmckaocywdkohmu",
"serializationFormat": "Json",
"schemaRef": "pknmdzqll",
"dataSources": [
"chkkpymxhp"
]
},
"builtInTransformationSettings": {
"serializationFormat": "Delta",
"schemaRef": "mcdc",
"datasets": [
{
"key": "qsfqcgxaxnhfumrsdsokwyv",
"description": "Lorem ipsum odor amet, consectetuer adipiscing elit.",
"schemaRef": "n",
"inputs": [
"mosffpsslifkq"
],
"expression": "aatbwomvflemsxialv"
}
],
"filter": [
{
"type": "Filter",
"description": "Lorem ipsum odor amet, consectetuer adipiscing elit.",
"inputs": [
"sxmjkbntgb"
],
"expression": "n"
}
],
"map": [
{
"type": "NewProperties",
"description": "Lorem ipsum odor amet, consectetuer adipiscing elit.",
"inputs": [
"xsbxuk"
],
"expression": "txoiltogsarwkzalsphvlmt",
"output": "nvgtmkfl"
}
]
},
"destinationSettings": {
"endpointRef": "kybkchnzimerguekuvqlqiqdvvrt",
"dataDestination": "cbrh"
}
}
],
"provisioningState": "Succeeded"
},
"extendedLocation": {
"name": "qmbrfwcpwwhggszhrdjv",
"type": "CustomLocation"
},
"id": "/subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/resourceGroup123/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123",
"name": "jxhcpwgfkxqasbexkookvxk",
"type": "zkuozvgjseokfchkscoswthzjdry",
"systemData": {
"createdBy": "ssvaslsmudloholronopqyxjcu",
"createdByType": "User",
"createdAt": "2024-08-09T18:13:29.389Z",
"lastModifiedBy": "gnicpuszwd",
"lastModifiedByType": "User",
"lastModifiedAt": "2024-08-09T18:13:29.389Z"
}
}Dataflow_CreateOrUpdate_ComplexContextualization
Sample request
PUT https://management.azure.com/subscriptions/F8C729F9-DF9C-4743-848F-96EE433D8E53/resourceGroups/rgiotoperations/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/aio-to-adx-contexualized?api-version=2024-11-01
{
"properties": {
"mode": "Enabled",
"operations": [
{
"operationType": "Source",
"name": "source1",
"sourceSettings": {
"endpointRef": "aio-builtin-broker-endpoint",
"dataSources": [
"azure-iot-operations/data/thermostat"
]
}
},
{
"operationType": "BuiltInTransformation",
"name": "transformation1",
"builtInTransformationSettings": {
"map": [
{
"inputs": [
"*"
],
"output": "*"
},
{
"inputs": [
"$context(quality).*"
],
"output": "enriched.*"
}
],
"datasets": [
{
"key": "quality",
"inputs": [
"$source.country",
"$context.country"
],
"expression": "$1 == $2"
}
]
}
},
{
"operationType": "Destination",
"name": "destination1",
"destinationSettings": {
"endpointRef": "adx-endpoint",
"dataDestination": "mytable"
}
}
]
},
"extendedLocation": {
"name": "qmbrfwcpwwhggszhrdjv",
"type": "CustomLocation"
}
}
Sample response
{
"properties": {
"mode": "Enabled",
"operations": [
{
"operationType": "Source",
"name": "source1",
"sourceSettings": {
"endpointRef": "aio-builtin-broker-endpoint",
"dataSources": [
"azure-iot-operations/data/thermostat"
]
}
},
{
"operationType": "BuiltInTransformation",
"name": "transformation1",
"builtInTransformationSettings": {
"map": [
{
"inputs": [
"*"
],
"output": "*"
},
{
"inputs": [
"$context(quality).*"
],
"output": "enriched.*"
}
],
"datasets": [
{
"key": "quality",
"inputs": [
"$source.country",
"$context.country"
],
"expression": "$1 == $2"
}
]
}
},
{
"operationType": "Destination",
"name": "destination1",
"destinationSettings": {
"endpointRef": "adx-endpoint",
"dataDestination": "mytable"
}
}
],
"provisioningState": "Succeeded"
},
"extendedLocation": {
"name": "qmbrfwcpwwhggszhrdjv",
"type": "CustomLocation"
},
"id": "/subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/resourceGroup123/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123",
"name": "jxhcpwgfkxqasbexkookvxk",
"type": "zkuozvgjseokfchkscoswthzjdry",
"systemData": {
"createdBy": "ssvaslsmudloholronopqyxjcu",
"createdByType": "User",
"createdAt": "2024-08-09T18:13:29.389Z",
"lastModifiedBy": "gnicpuszwd",
"lastModifiedByType": "User",
"lastModifiedAt": "2024-08-09T18:13:29.389Z"
}
}Azure-AsyncOperation: https://contoso.com/operationstatus{
"properties": {
"mode": "Enabled",
"operations": [
{
"operationType": "Source",
"name": "source1",
"sourceSettings": {
"endpointRef": "aio-builtin-broker-endpoint",
"dataSources": [
"azure-iot-operations/data/thermostat"
]
}
},
{
"operationType": "BuiltInTransformation",
"name": "transformation1",
"builtInTransformationSettings": {
"map": [
{
"inputs": [
"*"
],
"output": "*"
},
{
"inputs": [
"$context(quality).*"
],
"output": "enriched.*"
}
],
"datasets": [
{
"key": "quality",
"inputs": [
"$source.country",
"$context.country"
],
"expression": "$1 == $2"
}
]
}
},
{
"operationType": "Destination",
"name": "destination1",
"destinationSettings": {
"endpointRef": "adx-endpoint",
"dataDestination": "mytable"
}
}
],
"provisioningState": "Succeeded"
},
"extendedLocation": {
"name": "qmbrfwcpwwhggszhrdjv",
"type": "CustomLocation"
},
"id": "/subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/resourceGroup123/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123",
"name": "jxhcpwgfkxqasbexkookvxk",
"type": "zkuozvgjseokfchkscoswthzjdry",
"systemData": {
"createdBy": "ssvaslsmudloholronopqyxjcu",
"createdByType": "User",
"createdAt": "2024-08-09T18:13:29.389Z",
"lastModifiedBy": "gnicpuszwd",
"lastModifiedByType": "User",
"lastModifiedAt": "2024-08-09T18:13:29.389Z"
}
}Dataflow_CreateOrUpdate_ComplexEventHub
Sample request
PUT https://management.azure.com/subscriptions/F8C729F9-DF9C-4743-848F-96EE433D8E53/resourceGroups/rgiotoperations/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/aio-to-event-hub-transformed?api-version=2024-11-01
{
"properties": {
"mode": "Enabled",
"operations": [
{
"operationType": "Source",
"name": "source1",
"sourceSettings": {
"endpointRef": "aio-builtin-broker-endpoint",
"dataSources": [
"azure-iot-operations/data/thermostat"
]
}
},
{
"operationType": "BuiltInTransformation",
"builtInTransformationSettings": {
"filter": [
{
"inputs": [
"temperature.Value",
"\"Tag 10\".Value"
],
"expression": "$1 > 9000 && $2 >= 8000"
}
],
"map": [
{
"inputs": [
"*"
],
"output": "*"
},
{
"inputs": [
"temperature.Value",
"\"Tag 10\".Value"
],
"expression": "($1+$2)/2",
"output": "AvgTemp.Value"
},
{
"inputs": [],
"expression": "true",
"output": "dataflow-processed"
},
{
"inputs": [
"temperature.SourceTimestamp"
],
"expression": "",
"output": ""
},
{
"inputs": [
"\"Tag 10\""
],
"expression": "",
"output": "pressure"
},
{
"inputs": [
"temperature.Value"
],
"expression": "cToF($1)",
"output": "temperatureF.Value"
},
{
"inputs": [
"\"Tag 10\".Value"
],
"expression": "scale ($1,0,10,0,100)",
"output": "\"Scale Tag 10\".Value"
}
]
}
},
{
"operationType": "Destination",
"name": "destination1",
"destinationSettings": {
"endpointRef": "event-hub-endpoint",
"dataDestination": "myuniqueeventhub"
}
}
]
},
"extendedLocation": {
"name": "qmbrfwcpwwhggszhrdjv",
"type": "CustomLocation"
}
}
Sample response
{
"properties": {
"mode": "Enabled",
"operations": [
{
"operationType": "Source",
"name": "source1",
"sourceSettings": {
"endpointRef": "aio-builtin-broker-endpoint",
"dataSources": [
"azure-iot-operations/data/thermostat"
]
}
},
{
"operationType": "BuiltInTransformation",
"name": "transformation1",
"builtInTransformationSettings": {
"map": [
{
"inputs": [
"*"
],
"output": "*"
},
{
"inputs": [
"$context(quality).*"
],
"output": "enriched.*"
}
],
"datasets": [
{
"key": "quality",
"inputs": [
"$source.country",
"$context.country"
],
"expression": "$1 == $2"
}
]
}
},
{
"operationType": "Destination",
"name": "destination1",
"destinationSettings": {
"endpointRef": "adx-endpoint",
"dataDestination": "mytable"
}
}
],
"provisioningState": "Succeeded"
},
"extendedLocation": {
"name": "qmbrfwcpwwhggszhrdjv",
"type": "CustomLocation"
},
"id": "/subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/resourceGroup123/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123",
"name": "jxhcpwgfkxqasbexkookvxk",
"type": "zkuozvgjseokfchkscoswthzjdry",
"systemData": {
"createdBy": "ssvaslsmudloholronopqyxjcu",
"createdByType": "User",
"createdAt": "2024-08-09T18:13:29.389Z",
"lastModifiedBy": "gnicpuszwd",
"lastModifiedByType": "User",
"lastModifiedAt": "2024-08-09T18:13:29.389Z"
}
}Azure-AsyncOperation: https://contoso.com/operationstatus{
"properties": {
"mode": "Enabled",
"operations": [
{
"operationType": "Source",
"name": "source1",
"sourceSettings": {
"endpointRef": "aio-builtin-broker-endpoint",
"dataSources": [
"azure-iot-operations/data/thermostat"
]
}
},
{
"operationType": "BuiltInTransformation",
"name": "transformation1",
"builtInTransformationSettings": {
"map": [
{
"inputs": [
"*"
],
"output": "*"
},
{
"inputs": [
"$context(quality).*"
],
"output": "enriched.*"
}
],
"datasets": [
{
"key": "quality",
"inputs": [
"$source.country",
"$context.country"
],
"expression": "$1 == $2"
}
]
}
},
{
"operationType": "Destination",
"name": "destination1",
"destinationSettings": {
"endpointRef": "adx-endpoint",
"dataDestination": "mytable"
}
}
],
"provisioningState": "Succeeded"
},
"extendedLocation": {
"name": "qmbrfwcpwwhggszhrdjv",
"type": "CustomLocation"
},
"id": "/subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/resourceGroup123/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123",
"name": "jxhcpwgfkxqasbexkookvxk",
"type": "zkuozvgjseokfchkscoswthzjdry",
"systemData": {
"createdBy": "ssvaslsmudloholronopqyxjcu",
"createdByType": "User",
"createdAt": "2024-08-09T18:13:29.389Z",
"lastModifiedBy": "gnicpuszwd",
"lastModifiedByType": "User",
"lastModifiedAt": "2024-08-09T18:13:29.389Z"
}
}Dataflow_CreateOrUpdate_FilterToTopic
Sample request
PUT https://management.azure.com/subscriptions/F8C729F9-DF9C-4743-848F-96EE433D8E53/resourceGroups/rgiotoperations/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/mqtt-filter-to-topic?api-version=2024-11-01
{
"properties": {
"mode": "Enabled",
"operations": [
{
"operationType": "Source",
"name": "source1",
"sourceSettings": {
"endpointRef": "aio-builtin-broker-endpoint",
"dataSources": [
"azure-iot-operations/data/thermostat"
]
}
},
{
"operationType": "BuiltInTransformation",
"name": "transformation1",
"builtInTransformationSettings": {
"filter": [
{
"type": "Filter",
"description": "filter-datapoint",
"inputs": [
"temperature.Value",
"\"Tag 10\".Value"
],
"expression": "$1 > 9000 && $2 >= 8000"
}
],
"map": [
{
"type": "PassThrough",
"inputs": [
"*"
],
"output": "*"
}
]
}
},
{
"operationType": "Destination",
"name": "destination1",
"destinationSettings": {
"endpointRef": "aio-builtin-broker-endpoint",
"dataDestination": "data/filtered/thermostat"
}
}
]
},
"extendedLocation": {
"name": "qmbrfwcpwwhggszhrdjv",
"type": "CustomLocation"
}
}
Sample response
{
"properties": {
"mode": "Enabled",
"operations": [
{
"operationType": "Source",
"name": "source1",
"sourceSettings": {
"endpointRef": "aio-builtin-broker-endpoint",
"dataSources": [
"azure-iot-operations/data/thermostat"
]
}
},
{
"operationType": "BuiltInTransformation",
"name": "transformation1",
"builtInTransformationSettings": {
"filter": [
{
"type": "Filter",
"description": "filter-datapoint",
"inputs": [
"temperature.Value",
"\"Tag 10\".Value"
],
"expression": "$1 > 9000 && $2 >= 8000"
}
],
"map": [
{
"type": "PassThrough",
"inputs": [
"*"
],
"output": "*"
}
]
}
},
{
"operationType": "Destination",
"name": "destination1",
"destinationSettings": {
"endpointRef": "aio-builtin-broker-endpoint",
"dataDestination": "data/filtered/thermostat"
}
}
],
"provisioningState": "Succeeded"
},
"extendedLocation": {
"name": "qmbrfwcpwwhggszhrdjv",
"type": "CustomLocation"
},
"id": "/subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/resourceGroup123/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123",
"name": "jxhcpwgfkxqasbexkookvxk",
"type": "zkuozvgjseokfchkscoswthzjdry",
"systemData": {
"createdBy": "ssvaslsmudloholronopqyxjcu",
"createdByType": "User",
"createdAt": "2024-08-09T18:13:29.389Z",
"lastModifiedBy": "gnicpuszwd",
"lastModifiedByType": "User",
"lastModifiedAt": "2024-08-09T18:13:29.389Z"
}
}Azure-AsyncOperation: https://contoso.com/operationstatus{
"properties": {
"mode": "Enabled",
"operations": [
{
"operationType": "Source",
"name": "source1",
"sourceSettings": {
"endpointRef": "aio-builtin-broker-endpoint",
"dataSources": [
"azure-iot-operations/data/thermostat"
]
}
},
{
"operationType": "BuiltInTransformation",
"name": "transformation1",
"builtInTransformationSettings": {
"filter": [
{
"type": "Filter",
"description": "filter-datapoint",
"inputs": [
"temperature.Value",
"\"Tag 10\".Value"
],
"expression": "$1 > 9000 && $2 >= 8000"
}
],
"map": [
{
"type": "PassThrough",
"inputs": [
"*"
],
"output": "*"
}
]
}
},
{
"operationType": "Destination",
"name": "destination1",
"destinationSettings": {
"endpointRef": "aio-builtin-broker-endpoint",
"dataDestination": "data/filtered/thermostat"
}
}
],
"provisioningState": "Succeeded"
},
"extendedLocation": {
"name": "qmbrfwcpwwhggszhrdjv",
"type": "CustomLocation"
},
"id": "/subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/resourceGroup123/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123",
"name": "jxhcpwgfkxqasbexkookvxk",
"type": "zkuozvgjseokfchkscoswthzjdry",
"systemData": {
"createdBy": "ssvaslsmudloholronopqyxjcu",
"createdByType": "User",
"createdAt": "2024-08-09T18:13:29.389Z",
"lastModifiedBy": "gnicpuszwd",
"lastModifiedByType": "User",
"lastModifiedAt": "2024-08-09T18:13:29.389Z"
}
}Dataflow_CreateOrUpdate_SimpleEventGrid
Sample request
PUT https://management.azure.com/subscriptions/F8C729F9-DF9C-4743-848F-96EE433D8E53/resourceGroups/rgiotoperations/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/aio-to-event-grid?api-version=2024-11-01
{
"properties": {
"mode": "Enabled",
"operations": [
{
"operationType": "Source",
"name": "source1",
"sourceSettings": {
"endpointRef": "aio-builtin-broker-endpoint",
"dataSources": [
"thermostats/+/telemetry/temperature/#"
]
}
},
{
"operationType": "Destination",
"name": "destination1",
"destinationSettings": {
"endpointRef": "event-grid-endpoint",
"dataDestination": "factory/telemetry"
}
}
]
},
"extendedLocation": {
"name": "qmbrfwcpwwhggszhrdjv",
"type": "CustomLocation"
}
}
Sample response
{
"properties": {
"mode": "Enabled",
"operations": [
{
"operationType": "Source",
"name": "source1",
"sourceSettings": {
"endpointRef": "aio-builtin-broker-endpoint",
"dataSources": [
"thermostats/+/telemetry/temperature/#"
]
}
},
{
"operationType": "Destination",
"name": "destination1",
"destinationSettings": {
"endpointRef": "event-grid-endpoint",
"dataDestination": "factory/telemetry"
}
}
],
"provisioningState": "Succeeded"
},
"extendedLocation": {
"name": "qmbrfwcpwwhggszhrdjv",
"type": "CustomLocation"
},
"id": "/subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/resourceGroup123/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123",
"name": "jxhcpwgfkxqasbexkookvxk",
"type": "zkuozvgjseokfchkscoswthzjdry",
"systemData": {
"createdBy": "ssvaslsmudloholronopqyxjcu",
"createdByType": "User",
"createdAt": "2024-08-09T18:13:29.389Z",
"lastModifiedBy": "gnicpuszwd",
"lastModifiedByType": "User",
"lastModifiedAt": "2024-08-09T18:13:29.389Z"
}
}Azure-AsyncOperation: https://contoso.com/operationstatus{
"properties": {
"mode": "Enabled",
"operations": [
{
"operationType": "Source",
"name": "source1",
"sourceSettings": {
"endpointRef": "aio-builtin-broker-endpoint",
"dataSources": [
"thermostats/+/telemetry/temperature/#"
]
}
},
{
"operationType": "Destination",
"name": "destination1",
"destinationSettings": {
"endpointRef": "event-grid-endpoint",
"dataDestination": "factory/telemetry"
}
}
],
"provisioningState": "Succeeded"
},
"extendedLocation": {
"name": "qmbrfwcpwwhggszhrdjv",
"type": "CustomLocation"
},
"id": "/subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/resourceGroup123/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123",
"name": "jxhcpwgfkxqasbexkookvxk",
"type": "zkuozvgjseokfchkscoswthzjdry",
"systemData": {
"createdBy": "ssvaslsmudloholronopqyxjcu",
"createdByType": "User",
"createdAt": "2024-08-09T18:13:29.389Z",
"lastModifiedBy": "gnicpuszwd",
"lastModifiedByType": "User",
"lastModifiedAt": "2024-08-09T18:13:29.389Z"
}
}Dataflow_CreateOrUpdate_SimpleFabric
Sample request
PUT https://management.azure.com/subscriptions/F8C729F9-DF9C-4743-848F-96EE433D8E53/resourceGroups/rgiotoperations/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/aio-to-fabric?api-version=2024-11-01
{
"properties": {
"mode": "Enabled",
"operations": [
{
"operationType": "Source",
"name": "source1",
"sourceSettings": {
"endpointRef": "aio-builtin-broker-endpoint",
"dataSources": [
"azure-iot-operations/data/thermostat"
]
}
},
{
"operationType": "BuiltInTransformation",
"builtInTransformationSettings": {
"serializationFormat": "Parquet",
"schemaRef": "aio-sr://exampleNamespace/exmapleParquetSchema:1.0.0"
}
},
{
"operationType": "Destination",
"name": "destination1",
"destinationSettings": {
"endpointRef": "fabric-endpoint",
"dataDestination": "telemetryTable"
}
}
]
},
"extendedLocation": {
"name": "qmbrfwcpwwhggszhrdjv",
"type": "CustomLocation"
}
}
Sample response
{
"properties": {
"mode": "Enabled",
"operations": [
{
"operationType": "Source",
"name": "source1",
"sourceSettings": {
"endpointRef": "aio-builtin-broker-endpoint",
"dataSources": [
"azure-iot-operations/data/thermostat"
]
}
},
{
"operationType": "BuiltInTransformation",
"builtInTransformationSettings": {
"serializationFormat": "Parquet",
"schemaRef": "aio-sr://exampleNamespace/exmapleParquetSchema:1.0.0"
}
},
{
"operationType": "Destination",
"name": "destination1",
"destinationSettings": {
"endpointRef": "fabric-endpoint",
"dataDestination": "telemetryTable"
}
}
],
"provisioningState": "Succeeded"
},
"extendedLocation": {
"name": "qmbrfwcpwwhggszhrdjv",
"type": "CustomLocation"
},
"id": "/subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/resourceGroup123/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123",
"name": "jxhcpwgfkxqasbexkookvxk",
"type": "zkuozvgjseokfchkscoswthzjdry",
"systemData": {
"createdBy": "ssvaslsmudloholronopqyxjcu",
"createdByType": "User",
"createdAt": "2024-08-09T18:13:29.389Z",
"lastModifiedBy": "gnicpuszwd",
"lastModifiedByType": "User",
"lastModifiedAt": "2024-08-09T18:13:29.389Z"
}
}Azure-AsyncOperation: https://contoso.com/operationstatus{
"properties": {
"mode": "Enabled",
"operations": [
{
"operationType": "Source",
"name": "source1",
"sourceSettings": {
"endpointRef": "aio-builtin-broker-endpoint",
"dataSources": [
"azure-iot-operations/data/thermostat"
]
}
},
{
"operationType": "BuiltInTransformation",
"builtInTransformationSettings": {
"serializationFormat": "Parquet",
"schemaRef": "aio-sr://exampleNamespace/exmapleParquetSchema:1.0.0"
}
},
{
"operationType": "Destination",
"name": "destination1",
"destinationSettings": {
"endpointRef": "fabric-endpoint",
"dataDestination": "telemetryTable"
}
}
],
"provisioningState": "Succeeded"
},
"extendedLocation": {
"name": "qmbrfwcpwwhggszhrdjv",
"type": "CustomLocation"
},
"id": "/subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/resourceGroup123/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123",
"name": "jxhcpwgfkxqasbexkookvxk",
"type": "zkuozvgjseokfchkscoswthzjdry",
"systemData": {
"createdBy": "ssvaslsmudloholronopqyxjcu",
"createdByType": "User",
"createdAt": "2024-08-09T18:13:29.389Z",
"lastModifiedBy": "gnicpuszwd",
"lastModifiedByType": "User",
"lastModifiedAt": "2024-08-09T18:13:29.389Z"
}
}Definitions
| Name | Description |
|---|---|
|
created |
The type of identity that created the resource. |
|
Dataflow |
Dataflow BuiltIn Transformation dataset properties |
|
Dataflow |
Dataflow BuiltIn Transformation filter properties |
|
Dataflow |
Dataflow BuiltIn Transformation map properties |
|
Dataflow |
Dataflow BuiltIn Transformation properties |
|
Dataflow |
Dataflow Destination Operation properties |
|
Dataflow |
Dataflow type mapping properties |
|
Dataflow |
Dataflow Operation properties. NOTE - One only method is allowed to be used for one entry. |
|
Dataflow |
Dataflow Resource properties |
|
Dataflow |
Instance dataflowProfile dataflow resource |
|
Dataflow |
Dataflow Source Operation properties |
|
Error |
The resource management error additional info. |
|
Error |
The error detail. |
|
Error |
Error response |
|
Extended |
Extended location is an extension of Azure locations. They provide a way to use their Azure ARC enabled Kubernetes clusters as target locations for deploying Azure services instances. |
|
Extended |
The enum defining type of ExtendedLocation accepted. |
|
Filter |
Filter Type properties |
|
Operational |
Mode properties |
|
Operation |
Dataflow Operation Type properties |
|
Provisioning |
The enum defining status of resource. |
|
Source |
Serialization Format properties |
|
system |
Metadata pertaining to creation and last modification of the resource. |
|
Transformation |
Transformation Format properties |
createdByType
The type of identity that created the resource.
| Name | Type | Description |
|---|---|---|
| Application |
string |
|
| Key |
string |
|
| ManagedIdentity |
string |
|
| User |
string |
DataflowBuiltInTransformationDataset
Dataflow BuiltIn Transformation dataset properties
| Name | Type | Description |
|---|---|---|
| description |
string |
A user provided optional description of the dataset. |
| expression |
string |
Condition to enrich data from Broker State Store. Example: $1 < 0 || $1 > $2 (Assuming inputs section $1 and $2 are provided) |
| inputs |
string[] |
List of fields for enriching from the Broker State Store. |
| key |
string |
The key of the dataset. |
| schemaRef |
string |
The reference to the schema that describes the dataset. Allowed: JSON Schema/draft-7. |
DataflowBuiltInTransformationFilter
Dataflow BuiltIn Transformation filter properties
| Name | Type | Default value | Description |
|---|---|---|---|
| description |
string |
A user provided optional description of the filter. |
|
| expression |
string |
Condition to filter data. Can reference input fields with {n} where n is the index of the input field starting from 1. Example: $1 < 0 || $1 > $2 (Assuming inputs section $1 and $2 are provided) |
|
| inputs |
string[] |
List of fields for filtering in JSON path expression. |
|
| type | Filter |
The type of dataflow operation. |
DataflowBuiltInTransformationMap
Dataflow BuiltIn Transformation map properties
| Name | Type | Description |
|---|---|---|
| description |
string |
A user provided optional description of the mapping function. |
| expression |
string |
Modify the inputs field(s) to the final output field. Example: $1 * 2.2 (Assuming inputs section $1 is provided) |
| inputs |
string[] |
List of fields for mapping in JSON path expression. |
| output |
string |
Where and how the input fields to be organized in the output record. |
| type |
Type of transformation. |
DataflowBuiltInTransformationSettings
Dataflow BuiltIn Transformation properties
| Name | Type | Default value | Description |
|---|---|---|---|
| datasets | [] |
Enrich data from Broker State Store. Dataset references a key in Broker State Store. |
|
| filter | [] |
Filters input record or datapoints based on condition. |
|
| map | [] |
Maps input to output message. |
|
| schemaRef |
string |
Reference to the schema that describes the output of the transformation. |
|
| serializationFormat | Json |
Serialization format. Optional; defaults to JSON. Allowed value JSON Schema/draft-7, Parquet. Default: Json |
DataflowDestinationOperationSettings
Dataflow Destination Operation properties
| Name | Type | Description |
|---|---|---|
| dataDestination |
string |
Destination location, can be a topic or table name. Supports dynamic values with $topic, $systemProperties, $userProperties, $payload, $context, and $subscription. |
| endpointRef |
string |
Reference to the Endpoint CR. Can be of Broker, Kafka, Fabric, ADLS, ADX type. |
DataflowMappingType
Dataflow type mapping properties
| Name | Type | Description |
|---|---|---|
| BuiltInFunction |
string |
Built in function type |
| Compute |
string |
Compute type |
| NewProperties |
string |
New Properties type |
| PassThrough |
string |
Pass-through type |
| Rename |
string |
Rename type |
DataflowOperation
Dataflow Operation properties. NOTE - One only method is allowed to be used for one entry.
| Name | Type | Description |
|---|---|---|
| builtInTransformationSettings |
Built In Transformation configuration. |
|
| destinationSettings |
Destination configuration. |
|
| name |
string |
Optional user provided name of the transformation. |
| operationType |
Type of operation. |
|
| sourceSettings |
Source configuration. |
DataflowProperties
Dataflow Resource properties
| Name | Type | Default value | Description |
|---|---|---|---|
| mode | Enabled |
Mode for Dataflow. Optional; defaults to Enabled. |
|
| operations |
List of operations including source and destination references as well as transformation. |
||
| provisioningState |
The status of the last operation. |
DataflowResource
Instance dataflowProfile dataflow resource
| Name | Type | Description |
|---|---|---|
| extendedLocation |
Edge location of the resource. |
|
| id |
string |
Fully qualified resource ID for the resource. E.g. "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/{resourceProviderNamespace}/{resourceType}/{resourceName}" |
| name |
string |
The name of the resource |
| properties |
The resource-specific properties for this resource. |
|
| systemData |
Azure Resource Manager metadata containing createdBy and modifiedBy information. |
|
| type |
string |
The type of the resource. E.g. "Microsoft.Compute/virtualMachines" or "Microsoft.Storage/storageAccounts" |
DataflowSourceOperationSettings
Dataflow Source Operation properties
| Name | Type | Default value | Description |
|---|---|---|---|
| assetRef |
string |
Reference to the resource in Azure Device Registry where the data in the endpoint originates from. |
|
| dataSources |
string[] |
List of source locations. Can be Broker or Kafka topics. Supports wildcards # and +. |
|
| endpointRef |
string |
Reference to the Dataflow Endpoint resource. Can only be of Broker and Kafka type. |
|
| schemaRef |
string |
Schema CR reference. Data will be deserialized according to the schema, and dropped if it doesn't match. |
|
| serializationFormat | Json |
Content is a JSON Schema. Allowed: JSON Schema/draft-7. |
ErrorAdditionalInfo
The resource management error additional info.
| Name | Type | Description |
|---|---|---|
| info |
object |
The additional info. |
| type |
string |
The additional info type. |
ErrorDetail
The error detail.
| Name | Type | Description |
|---|---|---|
| additionalInfo |
The error additional info. |
|
| code |
string |
The error code. |
| details |
The error details. |
|
| message |
string |
The error message. |
| target |
string |
The error target. |
ErrorResponse
Error response
| Name | Type | Description |
|---|---|---|
| error |
The error object. |
ExtendedLocation
Extended location is an extension of Azure locations. They provide a way to use their Azure ARC enabled Kubernetes clusters as target locations for deploying Azure services instances.
| Name | Type | Description |
|---|---|---|
| name |
string |
The name of the extended location. |
| type |
Type of ExtendedLocation. |
ExtendedLocationType
The enum defining type of ExtendedLocation accepted.
| Name | Type | Description |
|---|---|---|
| CustomLocation |
string |
CustomLocation type |
FilterType
Filter Type properties
| Name | Type | Description |
|---|---|---|
| Filter |
string |
Filter type |
OperationalMode
Mode properties
| Name | Type | Description |
|---|---|---|
| Disabled |
string |
Disabled is equivalent to False. |
| Enabled |
string |
Enabled is equivalent to True |
OperationType
Dataflow Operation Type properties
| Name | Type | Description |
|---|---|---|
| BuiltInTransformation |
string |
Dataflow BuiltIn Transformation Operation |
| Destination |
string |
Dataflow Destination Operation |
| Source |
string |
Dataflow Source Operation |
ProvisioningState
The enum defining status of resource.
| Name | Type | Description |
|---|---|---|
| Accepted |
string |
Resource has been Accepted. |
| Canceled |
string |
Resource creation was canceled. |
| Deleting |
string |
Resource is Deleting. |
| Failed |
string |
Resource creation failed. |
| Provisioning |
string |
Resource is getting provisioned. |
| Succeeded |
string |
Resource has been created. |
| Updating |
string |
Resource is Updating. |
SourceSerializationFormat
Serialization Format properties
| Name | Type | Description |
|---|---|---|
| Json |
string |
JSON Format |
systemData
Metadata pertaining to creation and last modification of the resource.
| Name | Type | Description |
|---|---|---|
| createdAt |
string |
The timestamp of resource creation (UTC). |
| createdBy |
string |
The identity that created the resource. |
| createdByType |
The type of identity that created the resource. |
|
| lastModifiedAt |
string |
The timestamp of resource last modification (UTC) |
| lastModifiedBy |
string |
The identity that last modified the resource. |
| lastModifiedByType |
The type of identity that last modified the resource. |
TransformationSerializationFormat
Transformation Format properties
| Name | Type | Description |
|---|---|---|
| Delta |
string |
Delta Format |
| Json |
string |
JSON Format |
| Parquet |
string |
Parquet Format |