Microsoft.DataFactory factories
Bicep resource definition
The factories resource type can be deployed with operations that target:
- Resource groups - See resource group deployment commands
For a list of changed properties in each API version, see change log.
Resource format
To create a Microsoft.DataFactory/factories resource, add the following Bicep to your template.
resource symbolicname 'Microsoft.DataFactory/factories@2018-06-01' = {
name: 'string'
location: 'string'
tags: {
tagName1: 'tagValue1'
tagName2: 'tagValue2'
}
identity: {
type: 'string'
userAssignedIdentities: {
{customized property}: any()
}
}
properties: {
encryption: {
identity: {
userAssignedIdentity: 'string'
}
keyName: 'string'
keyVersion: 'string'
vaultBaseUrl: 'string'
}
globalParameters: {
{customized property}: {
type: 'string'
value: any()
}
}
publicNetworkAccess: 'string'
purviewConfiguration: {
purviewResourceId: 'string'
}
repoConfiguration: {
accountName: 'string'
collaborationBranch: 'string'
disablePublish: bool
lastCommitId: 'string'
repositoryName: 'string'
rootFolder: 'string'
type: 'string'
// For remaining properties, see FactoryRepoConfiguration objects
}
}
}
FactoryRepoConfiguration objects
Set the type property to specify the type of object.
For FactoryGitHubConfiguration, use:
type: 'FactoryGitHubConfiguration'
clientId: 'string'
clientSecret: {
byoaSecretAkvUrl: 'string'
byoaSecretName: 'string'
}
hostName: 'string'
For FactoryVSTSConfiguration, use:
type: 'FactoryVSTSConfiguration'
projectName: 'string'
tenantId: 'string'
Property values
factories
Name | Description | Value |
---|---|---|
name | The resource name | string (required) Character limit: 3-63 Valid characters: Alphanumerics and hyphens. Start and end with alphanumeric. Resource name must be unique across Azure. |
location | The resource location. | string |
tags | The resource tags. | Dictionary of tag names and values. See Tags in templates |
identity | Managed service identity of the factory. | FactoryIdentity |
properties | Properties of the factory. | FactoryProperties |
FactoryIdentity
Name | Description | Value |
---|---|---|
type | The identity type. | 'SystemAssigned' 'SystemAssigned,UserAssigned' 'UserAssigned' (required) |
userAssignedIdentities | List of user assigned identities for the factory. | UserAssignedIdentitiesDefinitionSpecification |
UserAssignedIdentitiesDefinitionSpecification
Name | Description | Value |
---|---|---|
{customized property} | For Bicep, you can use the any() function. |
FactoryProperties
Name | Description | Value |
---|---|---|
encryption | Properties to enable Customer Managed Key for the factory. | EncryptionConfiguration |
globalParameters | List of parameters for factory. | GlobalParameterDefinitionSpecification |
publicNetworkAccess | Whether or not public network access is allowed for the data factory. | 'Disabled' 'Enabled' |
purviewConfiguration | Purview information of the factory. | PurviewConfiguration |
repoConfiguration | Git repo information of the factory. | FactoryRepoConfiguration |
EncryptionConfiguration
Name | Description | Value |
---|---|---|
identity | User assigned identity to use to authenticate to customer's key vault. If not provided Managed Service Identity will be used. | CMKIdentityDefinition |
keyName | The name of the key in Azure Key Vault to use as Customer Managed Key. | string (required) |
keyVersion | The version of the key used for CMK. If not provided, latest version will be used. | string |
vaultBaseUrl | The url of the Azure Key Vault used for CMK. | string (required) |
CMKIdentityDefinition
Name | Description | Value |
---|---|---|
userAssignedIdentity | The resource id of the user assigned identity to authenticate to customer's key vault. | string |
GlobalParameterDefinitionSpecification
Name | Description | Value |
---|---|---|
{customized property} | GlobalParameterSpecification |
GlobalParameterSpecification
Name | Description | Value |
---|---|---|
type | Global Parameter type. | 'Array' 'Bool' 'Float' 'Int' 'Object' 'String' (required) |
value | Value of parameter. | For Bicep, you can use the any() function.(required) |
PurviewConfiguration
Name | Description | Value |
---|---|---|
purviewResourceId | Purview resource id. | string |
FactoryRepoConfiguration
Name | Description | Value |
---|---|---|
accountName | Account name. | string (required) |
collaborationBranch | Collaboration branch. | string (required) |
disablePublish | Disable manual publish operation in ADF studio to favor automated publish. | bool |
lastCommitId | Last commit id. | string |
repositoryName | Repository name. | string (required) |
rootFolder | Root folder. | string (required) |
type | Set the object type | FactoryGitHubConfiguration FactoryVSTSConfiguration (required) |
FactoryGitHubConfiguration
Name | Description | Value |
---|---|---|
type | Type of repo configuration. | 'FactoryGitHubConfiguration' (required) |
clientId | GitHub bring your own app client id. | string |
clientSecret | GitHub bring your own app client secret information. | GitHubClientSecret |
hostName | GitHub Enterprise host name. For example: https://github.mydomain.com |
string |
GitHubClientSecret
Name | Description | Value |
---|---|---|
byoaSecretAkvUrl | Bring your own app client secret AKV URL. | string |
byoaSecretName | Bring your own app client secret name in AKV. | string |
FactoryVstsConfiguration
Name | Description | Value |
---|---|---|
type | Type of repo configuration. | 'FactoryVSTSConfiguration' (required) |
projectName | VSTS project name. | string (required) |
tenantId | VSTS tenant id. | string |
Quickstart templates
The following quickstart templates deploy this resource type.
Template | Description |
---|---|
Deploy the Sports Analytics on Azure Architecture |
Creates an Azure storage account with ADLS Gen 2 enabled, an Azure Data Factory instance with linked services for the storage account (an the Azure SQL Database if deployed), and an Azure Databricks instance. The AAD identity for the user deploying the template and the managed identity for the ADF instance will be granted the Storage Blob Data Contributor role on the storage account. There are also options to deploy an Azure Key Vault instance, an Azure SQL Database, and an Azure Event Hub (for streaming use cases). When an Azure Key Vault is deployed, the data factory managed identity and the AAD identity for the user deploying the template will be granted the Key Vault Secrets User role. |
FinOps hub |
This template creates a new FinOps hub instance, including Data Lake storage and a Data Factory. |
More is possible with Azure Data Factory - One click to try Azure Data Factory |
This template creates a data factory pipeline for a copy activity from Azure Blob into another Azure Blob |
Create a Data Factory, copy from Salesforce to Blob |
This template uses Azure Data Factory to perform data movement from Salesforce to Azure Blobs |
Bulk copy using Azure Data Factory |
This template creates a V2 data factory that copies data in bulk an Azure SQL database to an Azure SQL data warehouse. |
Create a V2 data factory |
This template creates a V2 data factory that copies data from a folder in an Azure Blob Storage to another folder in the storage. |
Create a V2 data factory (MySQL) |
This template creates a V2 data factory that copies data from a folder in an Azure Blob Storage to a table in an Azure Database for MySQL. |
Create a V2 data factory (PostGre) |
This template creates a V2 data factory that copies data from a folder in an Azure Blob Storage to a table in an Azure Database for PostgreSQL. |
Create a V2 data factory (SQL) |
This template creates a V2 data factory that copies data from a folder in an Azure Blob Storage to a table in an Azure SQL database. |
Create an empty data factory |
This template creates an empty data factory |
Azure Data Factory with Git and managed vnet configuration |
This template creates Azure Data Factory with Git configuration and managed virtual network. |
Create a V2 data factory (SQL On-prem) |
This template creates a V2 data factory that copies data from an on-premises SQL Server to an Azure blob storage |
Provision SSIS runtime in Azure |
This template creates a V2 data factory and then provisions an Azure SSIS integration runtime |
Create a V2 data factory (Spark) |
This template creates a V2 data factory that copies data from a folder in an Azure Blob Storage to another folder in the storage. |
ARM template resource definition
The factories resource type can be deployed with operations that target:
- Resource groups - See resource group deployment commands
For a list of changed properties in each API version, see change log.
Resource format
To create a Microsoft.DataFactory/factories resource, add the following JSON to your template.
{
"type": "Microsoft.DataFactory/factories",
"apiVersion": "2018-06-01",
"name": "string",
"location": "string",
"tags": {
"tagName1": "tagValue1",
"tagName2": "tagValue2"
},
"identity": {
"type": "string",
"userAssignedIdentities": {
"{customized property}": {}
}
},
"properties": {
"encryption": {
"identity": {
"userAssignedIdentity": "string"
},
"keyName": "string",
"keyVersion": "string",
"vaultBaseUrl": "string"
},
"globalParameters": {
"{customized property}": {
"type": "string",
"value": {}
}
},
"publicNetworkAccess": "string",
"purviewConfiguration": {
"purviewResourceId": "string"
},
"repoConfiguration": {
"accountName": "string",
"collaborationBranch": "string",
"disablePublish": "bool",
"lastCommitId": "string",
"repositoryName": "string",
"rootFolder": "string",
"type": "string"
// For remaining properties, see FactoryRepoConfiguration objects
}
}
}
FactoryRepoConfiguration objects
Set the type property to specify the type of object.
For FactoryGitHubConfiguration, use:
"type": "FactoryGitHubConfiguration",
"clientId": "string",
"clientSecret": {
"byoaSecretAkvUrl": "string",
"byoaSecretName": "string"
},
"hostName": "string"
For FactoryVSTSConfiguration, use:
"type": "FactoryVSTSConfiguration",
"projectName": "string",
"tenantId": "string"
Property values
factories
Name | Description | Value |
---|---|---|
type | The resource type | 'Microsoft.DataFactory/factories' |
apiVersion | The resource api version | '2018-06-01' |
name | The resource name | string (required) Character limit: 3-63 Valid characters: Alphanumerics and hyphens. Start and end with alphanumeric. Resource name must be unique across Azure. |
location | The resource location. | string |
tags | The resource tags. | Dictionary of tag names and values. See Tags in templates |
identity | Managed service identity of the factory. | FactoryIdentity |
properties | Properties of the factory. | FactoryProperties |
FactoryIdentity
Name | Description | Value |
---|---|---|
type | The identity type. | 'SystemAssigned' 'SystemAssigned,UserAssigned' 'UserAssigned' (required) |
userAssignedIdentities | List of user assigned identities for the factory. | UserAssignedIdentitiesDefinitionSpecification |
UserAssignedIdentitiesDefinitionSpecification
Name | Description | Value |
---|---|---|
{customized property} |
FactoryProperties
Name | Description | Value |
---|---|---|
encryption | Properties to enable Customer Managed Key for the factory. | EncryptionConfiguration |
globalParameters | List of parameters for factory. | GlobalParameterDefinitionSpecification |
publicNetworkAccess | Whether or not public network access is allowed for the data factory. | 'Disabled' 'Enabled' |
purviewConfiguration | Purview information of the factory. | PurviewConfiguration |
repoConfiguration | Git repo information of the factory. | FactoryRepoConfiguration |
EncryptionConfiguration
Name | Description | Value |
---|---|---|
identity | User assigned identity to use to authenticate to customer's key vault. If not provided Managed Service Identity will be used. | CMKIdentityDefinition |
keyName | The name of the key in Azure Key Vault to use as Customer Managed Key. | string (required) |
keyVersion | The version of the key used for CMK. If not provided, latest version will be used. | string |
vaultBaseUrl | The url of the Azure Key Vault used for CMK. | string (required) |
CMKIdentityDefinition
Name | Description | Value |
---|---|---|
userAssignedIdentity | The resource id of the user assigned identity to authenticate to customer's key vault. | string |
GlobalParameterDefinitionSpecification
Name | Description | Value |
---|---|---|
{customized property} | GlobalParameterSpecification |
GlobalParameterSpecification
Name | Description | Value |
---|---|---|
type | Global Parameter type. | 'Array' 'Bool' 'Float' 'Int' 'Object' 'String' (required) |
value | Value of parameter. |
PurviewConfiguration
Name | Description | Value |
---|---|---|
purviewResourceId | Purview resource id. | string |
FactoryRepoConfiguration
Name | Description | Value |
---|---|---|
accountName | Account name. | string (required) |
collaborationBranch | Collaboration branch. | string (required) |
disablePublish | Disable manual publish operation in ADF studio to favor automated publish. | bool |
lastCommitId | Last commit id. | string |
repositoryName | Repository name. | string (required) |
rootFolder | Root folder. | string (required) |
type | Set the object type | FactoryGitHubConfiguration FactoryVSTSConfiguration (required) |
FactoryGitHubConfiguration
Name | Description | Value |
---|---|---|
type | Type of repo configuration. | 'FactoryGitHubConfiguration' (required) |
clientId | GitHub bring your own app client id. | string |
clientSecret | GitHub bring your own app client secret information. | GitHubClientSecret |
hostName | GitHub Enterprise host name. For example: https://github.mydomain.com |
string |
GitHubClientSecret
Name | Description | Value |
---|---|---|
byoaSecretAkvUrl | Bring your own app client secret AKV URL. | string |
byoaSecretName | Bring your own app client secret name in AKV. | string |
FactoryVstsConfiguration
Name | Description | Value |
---|---|---|
type | Type of repo configuration. | 'FactoryVSTSConfiguration' (required) |
projectName | VSTS project name. | string (required) |
tenantId | VSTS tenant id. | string |
Quickstart templates
The following quickstart templates deploy this resource type.
Template | Description |
---|---|
Deploy the Sports Analytics on Azure Architecture |
Creates an Azure storage account with ADLS Gen 2 enabled, an Azure Data Factory instance with linked services for the storage account (an the Azure SQL Database if deployed), and an Azure Databricks instance. The AAD identity for the user deploying the template and the managed identity for the ADF instance will be granted the Storage Blob Data Contributor role on the storage account. There are also options to deploy an Azure Key Vault instance, an Azure SQL Database, and an Azure Event Hub (for streaming use cases). When an Azure Key Vault is deployed, the data factory managed identity and the AAD identity for the user deploying the template will be granted the Key Vault Secrets User role. |
FinOps hub |
This template creates a new FinOps hub instance, including Data Lake storage and a Data Factory. |
More is possible with Azure Data Factory - One click to try Azure Data Factory |
This template creates a data factory pipeline for a copy activity from Azure Blob into another Azure Blob |
Create a Data Factory, copy from Salesforce to Blob |
This template uses Azure Data Factory to perform data movement from Salesforce to Azure Blobs |
Bulk copy using Azure Data Factory |
This template creates a V2 data factory that copies data in bulk an Azure SQL database to an Azure SQL data warehouse. |
Create a V2 data factory |
This template creates a V2 data factory that copies data from a folder in an Azure Blob Storage to another folder in the storage. |
Create a V2 data factory (MySQL) |
This template creates a V2 data factory that copies data from a folder in an Azure Blob Storage to a table in an Azure Database for MySQL. |
Create a V2 data factory (PostGre) |
This template creates a V2 data factory that copies data from a folder in an Azure Blob Storage to a table in an Azure Database for PostgreSQL. |
Create a V2 data factory (SQL) |
This template creates a V2 data factory that copies data from a folder in an Azure Blob Storage to a table in an Azure SQL database. |
Create an empty data factory |
This template creates an empty data factory |
Azure Data Factory with Git and managed vnet configuration |
This template creates Azure Data Factory with Git configuration and managed virtual network. |
Create a V2 data factory (SQL On-prem) |
This template creates a V2 data factory that copies data from an on-premises SQL Server to an Azure blob storage |
Provision SSIS runtime in Azure |
This template creates a V2 data factory and then provisions an Azure SSIS integration runtime |
Create a V2 data factory (Spark) |
This template creates a V2 data factory that copies data from a folder in an Azure Blob Storage to another folder in the storage. |
Terraform (AzAPI provider) resource definition
The factories resource type can be deployed with operations that target:
- Resource groups
For a list of changed properties in each API version, see change log.
Resource format
To create a Microsoft.DataFactory/factories resource, add the following Terraform to your template.
resource "azapi_resource" "symbolicname" {
type = "Microsoft.DataFactory/factories@2018-06-01"
name = "string"
location = "string"
parent_id = "string"
tags = {
tagName1 = "tagValue1"
tagName2 = "tagValue2"
}
identity {
type = "string"
identity_ids = []
}
body = jsonencode({
properties = {
encryption = {
identity = {
userAssignedIdentity = "string"
}
keyName = "string"
keyVersion = "string"
vaultBaseUrl = "string"
}
globalParameters = {
{customized property} = {
type = "string"
}
}
publicNetworkAccess = "string"
purviewConfiguration = {
purviewResourceId = "string"
}
repoConfiguration = {
accountName = "string"
collaborationBranch = "string"
disablePublish = bool
lastCommitId = "string"
repositoryName = "string"
rootFolder = "string"
type = "string"
// For remaining properties, see FactoryRepoConfiguration objects
}
}
})
}
FactoryRepoConfiguration objects
Set the type property to specify the type of object.
For FactoryGitHubConfiguration, use:
type = "FactoryGitHubConfiguration"
clientId = "string"
clientSecret = {
byoaSecretAkvUrl = "string"
byoaSecretName = "string"
}
hostName = "string"
For FactoryVSTSConfiguration, use:
type = "FactoryVSTSConfiguration"
projectName = "string"
tenantId = "string"
Property values
factories
Name | Description | Value |
---|---|---|
type | The resource type | "Microsoft.DataFactory/factories@2018-06-01" |
name | The resource name | string (required) Character limit: 3-63 Valid characters: Alphanumerics and hyphens. Start and end with alphanumeric. Resource name must be unique across Azure. |
location | The resource location. | string |
parent_id | To deploy to a resource group, use the ID of that resource group. | string (required) |
tags | The resource tags. | Dictionary of tag names and values. |
identity | Managed service identity of the factory. | FactoryIdentity |
properties | Properties of the factory. | FactoryProperties |
FactoryIdentity
Name | Description | Value |
---|---|---|
type | The identity type. | "SystemAssigned" "SystemAssigned,UserAssigned" "UserAssigned" (required) |
identity_ids | List of user assigned identities for the factory. | Array of user identity IDs. |
UserAssignedIdentitiesDefinitionSpecification
Name | Description | Value |
---|---|---|
{customized property} |
FactoryProperties
Name | Description | Value |
---|---|---|
encryption | Properties to enable Customer Managed Key for the factory. | EncryptionConfiguration |
globalParameters | List of parameters for factory. | GlobalParameterDefinitionSpecification |
publicNetworkAccess | Whether or not public network access is allowed for the data factory. | "Disabled" "Enabled" |
purviewConfiguration | Purview information of the factory. | PurviewConfiguration |
repoConfiguration | Git repo information of the factory. | FactoryRepoConfiguration |
EncryptionConfiguration
Name | Description | Value |
---|---|---|
identity | User assigned identity to use to authenticate to customer's key vault. If not provided Managed Service Identity will be used. | CMKIdentityDefinition |
keyName | The name of the key in Azure Key Vault to use as Customer Managed Key. | string (required) |
keyVersion | The version of the key used for CMK. If not provided, latest version will be used. | string |
vaultBaseUrl | The url of the Azure Key Vault used for CMK. | string (required) |
CMKIdentityDefinition
Name | Description | Value |
---|---|---|
userAssignedIdentity | The resource id of the user assigned identity to authenticate to customer's key vault. | string |
GlobalParameterDefinitionSpecification
Name | Description | Value |
---|---|---|
{customized property} | GlobalParameterSpecification |
GlobalParameterSpecification
Name | Description | Value |
---|---|---|
type | Global Parameter type. | "Array" "Bool" "Float" "Int" "Object" "String" (required) |
value | Value of parameter. |
PurviewConfiguration
Name | Description | Value |
---|---|---|
purviewResourceId | Purview resource id. | string |
FactoryRepoConfiguration
Name | Description | Value |
---|---|---|
accountName | Account name. | string (required) |
collaborationBranch | Collaboration branch. | string (required) |
disablePublish | Disable manual publish operation in ADF studio to favor automated publish. | bool |
lastCommitId | Last commit id. | string |
repositoryName | Repository name. | string (required) |
rootFolder | Root folder. | string (required) |
type | Set the object type | FactoryGitHubConfiguration FactoryVSTSConfiguration (required) |
FactoryGitHubConfiguration
Name | Description | Value |
---|---|---|
type | Type of repo configuration. | "FactoryGitHubConfiguration" (required) |
clientId | GitHub bring your own app client id. | string |
clientSecret | GitHub bring your own app client secret information. | GitHubClientSecret |
hostName | GitHub Enterprise host name. For example: https://github.mydomain.com |
string |
GitHubClientSecret
Name | Description | Value |
---|---|---|
byoaSecretAkvUrl | Bring your own app client secret AKV URL. | string |
byoaSecretName | Bring your own app client secret name in AKV. | string |
FactoryVstsConfiguration
Name | Description | Value |
---|---|---|
type | Type of repo configuration. | "FactoryVSTSConfiguration" (required) |
projectName | VSTS project name. | string (required) |
tenantId | VSTS tenant id. | string |