Microsoft.DataFactory factories/datasets 2017-09-01-preview

Article
03/13/2024

Bicep resource definition

The factories/datasets resource type can be deployed with operations that target:

Resource groups - See resource group deployment commands

For a list of changed properties in each API version, see change log.

Resource format

To create a Microsoft.DataFactory/factories/datasets resource, add the following Bicep to your template.

resource symbolicname 'Microsoft.DataFactory/factories/datasets@2017-09-01-preview' = {
  name: 'string'
  parent: resourceSymbolicName
  properties: {
    annotations: [
      any
    ]
    description: 'string'
    linkedServiceName: {
      parameters: {
        {customized property}: any()
      }
      referenceName: 'string'
      type: 'LinkedServiceReference'
    }
    parameters: {
      {customized property}: {
        defaultValue: any()
        type: 'string'
      }
    }
    structure: any()
    type: 'string'
    // For remaining properties, see Dataset objects
  }
}

Dataset objects

Set the type property to specify the type of object.

For AmazonMWSObject, use:

  type: 'AmazonMWSObject'

For AmazonS3Object, use:

  type: 'AmazonS3Object'
  typeProperties: {
    bucketName: any()
    compression: {
      type: 'string'
      // For remaining properties, see DatasetCompression objects
    }
    format: {
      deserializer: any()
      serializer: any()
      // For remaining properties, see DatasetStorageFormat objects
    }
    key: any()
    prefix: any()
    version: any()
  }

For AzureBlob, use:

  type: 'AzureBlob'
  typeProperties: {
    compression: {
      type: 'string'
      // For remaining properties, see DatasetCompression objects
    }
    fileName: any()
    folderPath: any()
    format: {
      deserializer: any()
      serializer: any()
      // For remaining properties, see DatasetStorageFormat objects
    }
    tableRootLocation: any()
  }

For AzureDataLakeStoreFile, use:

  type: 'AzureDataLakeStoreFile'
  typeProperties: {
    compression: {
      type: 'string'
      // For remaining properties, see DatasetCompression objects
    }
    fileName: any()
    folderPath: any()
    format: {
      deserializer: any()
      serializer: any()
      // For remaining properties, see DatasetStorageFormat objects
    }
  }

For AzureMySqlTable, use:

  type: 'AzureMySqlTable'
  typeProperties: {
    tableName: any()
  }

For AzurePostgreSqlTable, use:

  type: 'AzurePostgreSqlTable'

For AzureSearchIndex, use:

  type: 'AzureSearchIndex'
  typeProperties: {
    indexName: any()
  }

For AzureSqlDWTable, use:

  type: 'AzureSqlDWTable'
  typeProperties: {
    tableName: any()
  }

For AzureSqlTable, use:

  type: 'AzureSqlTable'
  typeProperties: {
    tableName: any()
  }

For AzureTable, use:

  type: 'AzureTable'
  typeProperties: {
    tableName: any()
  }

For CassandraTable, use:

  type: 'CassandraTable'
  typeProperties: {
    keyspace: any()
    tableName: any()
  }

For ConcurObject, use:

  type: 'ConcurObject'

For CouchbaseTable, use:

  type: 'CouchbaseTable'

For DocumentDbCollection, use:

  type: 'DocumentDbCollection'
  typeProperties: {
    collectionName: any()
  }

For DrillTable, use:

  type: 'DrillTable'

For DynamicsEntity, use:

  type: 'DynamicsEntity'
  typeProperties: {
    entityName: any()
  }

For EloquaObject, use:

  type: 'EloquaObject'

For FileShare, use:

  type: 'FileShare'
  typeProperties: {
    compression: {
      type: 'string'
      // For remaining properties, see DatasetCompression objects
    }
    fileFilter: any()
    fileName: any()
    folderPath: any()
    format: {
      deserializer: any()
      serializer: any()
      // For remaining properties, see DatasetStorageFormat objects
    }
  }

For GoogleBigQueryObject, use:

  type: 'GoogleBigQueryObject'

For GreenplumTable, use:

  type: 'GreenplumTable'

For HBaseObject, use:

  type: 'HBaseObject'

For HiveObject, use:

  type: 'HiveObject'

For HttpFile, use:

  type: 'HttpFile'
  typeProperties: {
    additionalHeaders: any()
    compression: {
      type: 'string'
      // For remaining properties, see DatasetCompression objects
    }
    format: {
      deserializer: any()
      serializer: any()
      // For remaining properties, see DatasetStorageFormat objects
    }
    relativeUrl: any()
    requestBody: any()
    requestMethod: any()
  }

For HubspotObject, use:

  type: 'HubspotObject'

For ImpalaObject, use:

  type: 'ImpalaObject'

For JiraObject, use:

  type: 'JiraObject'

For MagentoObject, use:

  type: 'MagentoObject'

For MariaDBTable, use:

  type: 'MariaDBTable'

For MarketoObject, use:

  type: 'MarketoObject'

For MongoDbCollection, use:

  type: 'MongoDbCollection'
  typeProperties: {
    collectionName: any()
  }

For NetezzaTable, use:

  type: 'NetezzaTable'

For ODataResource, use:

  type: 'ODataResource'
  typeProperties: {
    path: any()
  }

For OracleTable, use:

  type: 'OracleTable'
  typeProperties: {
    tableName: any()
  }

For PaypalObject, use:

  type: 'PaypalObject'

For PhoenixObject, use:

  type: 'PhoenixObject'

For PrestoObject, use:

  type: 'PrestoObject'

For QuickBooksObject, use:

  type: 'QuickBooksObject'

For RelationalTable, use:

  type: 'RelationalTable'
  typeProperties: {
    tableName: any()
  }

For ResponsysObject, use:

  type: 'ResponsysObject'

For SalesforceMarketingCloudObject, use:

  type: 'SalesforceMarketingCloudObject'

For SalesforceObject, use:

  type: 'SalesforceObject'
  typeProperties: {
    objectApiName: any()
  }

For SapCloudForCustomerResource, use:

  type: 'SapCloudForCustomerResource'
  typeProperties: {
    path: any()
  }

For SapEccResource, use:

  type: 'SapEccResource'
  typeProperties: {
    path: any()
  }

For ServiceNowObject, use:

  type: 'ServiceNowObject'

For ShopifyObject, use:

  type: 'ShopifyObject'

For SparkObject, use:

  type: 'SparkObject'

For SqlServerTable, use:

  type: 'SqlServerTable'
  typeProperties: {
    tableName: any()
  }

For SquareObject, use:

  type: 'SquareObject'

For VerticaTable, use:

  type: 'VerticaTable'

For WebTable, use:

  type: 'WebTable'
  typeProperties: {
    index: any()
    path: any()
  }

For XeroObject, use:

  type: 'XeroObject'

For ZohoObject, use:

  type: 'ZohoObject'

DatasetCompression objects

Set the type property to specify the type of object.

For BZip2, use:

  type: 'BZip2'

For Deflate, use:

  type: 'Deflate'
  level: 'string'

For GZip, use:

  type: 'GZip'
  level: 'string'

For ZipDeflate, use:

  type: 'ZipDeflate'
  level: 'string'

DatasetStorageFormat objects

Set the type property to specify the type of object.

Property values

factories/datasets

Name	Description	Value
name	The resource name See how to set names and types for child resources in Bicep.	string (required) Character limit: 1-260 Valid characters: Can't use: `<>*#.%&:\\+?/-` or control characters Start with alphanumeric.
parent	In Bicep, you can specify the parent resource for a child resource. You only need to add this property when the child resource is declared outside of the parent resource. For more information, see Child resource outside parent resource.	Symbolic name for resource of type: factories
properties	Dataset properties.	Dataset (required)

Dataset

Name	Description	Value
annotations	List of tags that can be used for describing the Dataset.	any[]
description	Dataset description.	string
linkedServiceName	Linked service reference.	LinkedServiceReference (required)
parameters	Parameters for dataset.	ParameterDefinitionSpecification
structure	Columns that define the structure of the dataset. Type: array (or Expression with resultType array), itemType: DatasetDataElement.	For Bicep, you can use the any() function.
type	Set the object type	AmazonMWSObject AmazonS3Object AzureBlob AzureDataLakeStoreFile AzureMySqlTable AzurePostgreSqlTable AzureSearchIndex AzureSqlDWTable AzureSqlTable AzureTable CassandraTable ConcurObject CouchbaseTable DocumentDbCollection DrillTable DynamicsEntity EloquaObject FileShare GoogleBigQueryObject GreenplumTable HBaseObject HiveObject HttpFile HubspotObject ImpalaObject JiraObject MagentoObject MariaDBTable MarketoObject MongoDbCollection NetezzaTable ODataResource OracleTable PaypalObject PhoenixObject PrestoObject QuickBooksObject RelationalTable ResponsysObject SalesforceMarketingCloudObject SalesforceObject SapCloudForCustomerResource SapEccResource ServiceNowObject ShopifyObject SparkObject SqlServerTable SquareObject VerticaTable WebTable XeroObject ZohoObject (required)

LinkedServiceReference

Name	Description	Value
parameters	Arguments for LinkedService.	ParameterValueSpecification
referenceName	Reference LinkedService name.	string (required)
type	Linked service reference type.	'LinkedServiceReference' (required)

ParameterValueSpecification

Name	Description	Value
{customized property}		For Bicep, you can use the any() function.

ParameterDefinitionSpecification

Name	Description	Value
{customized property}		ParameterSpecification

ParameterSpecification

Name	Description	Value
defaultValue	Default value of parameter.	For Bicep, you can use the any() function.
type	Parameter type.	'Array' 'Bool' 'Float' 'Int' 'Object' 'SecureString' 'String' (required)

AmazonMWSObjectDataset

Name	Description	Value
type	Type of dataset.	'AmazonMWSObject' (required)

AmazonS3Dataset

Name	Description	Value
type	Type of dataset.	'AmazonS3Object' (required)
typeProperties	Amazon S3 dataset properties.	AmazonS3DatasetTypeProperties (required)

AmazonS3DatasetTypeProperties

Name	Description	Value
bucketName	The name of the Amazon S3 bucket. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.(required)
compression	The data compression method used for the Amazon S3 object.	DatasetCompression
format	The format of files.	DatasetStorageFormat
key	The key of the Amazon S3 object. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.
prefix	The prefix filter for the S3 object name. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.
version	The version for the S3 object. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.

DatasetCompression

Name	Description	Value
type	Set the object type	BZip2 Deflate GZip ZipDeflate (required)

DatasetBZip2Compression

Name	Description	Value
type	Type of dataset compression.	'BZip2' (required)

DatasetDeflateCompression

Name	Description	Value
type	Type of dataset compression.	'Deflate' (required)
level	The Deflate compression level.	'Fastest' 'Optimal'

DatasetGZipCompression

Name	Description	Value
type	Type of dataset compression.	'GZip' (required)
level	The GZip compression level.	'Fastest' 'Optimal'

DatasetZipDeflateCompression

Name	Description	Value
type	Type of dataset compression.	'ZipDeflate' (required)
level	The ZipDeflate compression level.	'Fastest' 'Optimal'

DatasetStorageFormat

Name	Description	Value
deserializer	Deserializer. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.
serializer	Serializer. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.

AzureBlobDataset

Name	Description	Value
type	Type of dataset.	'AzureBlob' (required)
typeProperties	Azure Blob dataset properties.	AzureBlobDatasetTypeProperties (required)

AzureBlobDatasetTypeProperties

Name	Description	Value
compression	The data compression method used for the blob storage.	DatasetCompression
fileName	The name of the Azure Blob. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.
folderPath	The path of the Azure Blob storage. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.
format	The format of the Azure Blob storage.	DatasetStorageFormat
tableRootLocation	The root of blob path. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.

AzureDataLakeStoreDataset

Name	Description	Value
type	Type of dataset.	'AzureDataLakeStoreFile' (required)
typeProperties	Azure Data Lake Store dataset properties.	AzureDataLakeStoreDatasetTypeProperties (required)

AzureDataLakeStoreDatasetTypeProperties

Name	Description	Value
compression	The data compression method used for the item(s) in the Azure Data Lake Store.	DatasetCompression
fileName	The name of the file in the Azure Data Lake Store. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.
folderPath	Path to the folder in the Azure Data Lake Store. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.(required)
format	The format of the Data Lake Store.	DatasetStorageFormat

AzureMySqlTableDataset

Name	Description	Value
type	Type of dataset.	'AzureMySqlTable' (required)
typeProperties	Azure MySQL database dataset properties.	AzureMySqlTableDatasetTypeProperties (required)

AzureMySqlTableDatasetTypeProperties

Name	Description	Value
tableName	The Azure MySQL database table name. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.

AzurePostgreSqlTableDataset

Name	Description	Value
type	Type of dataset.	'AzurePostgreSqlTable' (required)

AzureSearchIndexDataset

Name	Description	Value
type	Type of dataset.	'AzureSearchIndex' (required)
typeProperties	Properties specific to this dataset type.	AzureSearchIndexDatasetTypeProperties (required)

AzureSearchIndexDatasetTypeProperties

Name	Description	Value
indexName	The name of the Azure Search Index. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.(required)

AzureSqlDWTableDataset

Name	Description	Value
type	Type of dataset.	'AzureSqlDWTable' (required)
typeProperties	Azure SQL Data Warehouse dataset properties.	AzureSqlDWTableDatasetTypeProperties (required)

AzureSqlDWTableDatasetTypeProperties

Name	Description	Value
tableName	The table name of the Azure SQL Data Warehouse. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.(required)

AzureSqlTableDataset

Name	Description	Value
type	Type of dataset.	'AzureSqlTable' (required)
typeProperties	Azure SQL dataset properties.	AzureSqlTableDatasetTypeProperties (required)

AzureSqlTableDatasetTypeProperties

Name	Description	Value
tableName	The table name of the Azure SQL database. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.(required)

AzureTableDataset

Name	Description	Value
type	Type of dataset.	'AzureTable' (required)
typeProperties	Azure Table dataset properties.	AzureTableDatasetTypeProperties (required)

AzureTableDatasetTypeProperties

Name	Description	Value
tableName	The table name of the Azure Table storage. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.(required)

CassandraTableDataset

Name	Description	Value
type	Type of dataset.	'CassandraTable' (required)
typeProperties	Cassandra dataset properties.	CassandraTableDatasetTypeProperties (required)

CassandraTableDatasetTypeProperties

Name	Description	Value
keyspace	The keyspace of the Cassandra database. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.
tableName	The table name of the Cassandra database. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.

ConcurObjectDataset

Name	Description	Value
type	Type of dataset.	'ConcurObject' (required)

CouchbaseTableDataset

Name	Description	Value
type	Type of dataset.	'CouchbaseTable' (required)

DocumentDbCollectionDataset

Name	Description	Value
type	Type of dataset.	'DocumentDbCollection' (required)
typeProperties	DocumentDB Collection dataset properties.	DocumentDbCollectionDatasetTypeProperties (required)

DocumentDbCollectionDatasetTypeProperties

Name	Description	Value
collectionName	Document Database collection name. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.(required)

DrillTableDataset

Name	Description	Value
type	Type of dataset.	'DrillTable' (required)

DynamicsEntityDataset

Name	Description	Value
type	Type of dataset.	'DynamicsEntity' (required)
typeProperties	Dynamics entity dataset properties.	DynamicsEntityDatasetTypeProperties (required)

DynamicsEntityDatasetTypeProperties

Name	Description	Value
entityName	The logical name of the entity. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.

EloquaObjectDataset

Name	Description	Value
type	Type of dataset.	'EloquaObject' (required)

FileShareDataset

Name	Description	Value
type	Type of dataset.	'FileShare' (required)
typeProperties	On-premises file system dataset properties.	FileShareDatasetTypeProperties (required)

FileShareDatasetTypeProperties

Name	Description	Value
compression	The data compression method used for the file system.	DatasetCompression
fileFilter	Specify a filter to be used to select a subset of files in the folderPath rather than all files. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.
fileName	The name of the on-premises file system. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.
folderPath	The path of the on-premises file system. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.
format	The format of the files.	DatasetStorageFormat

GoogleBigQueryObjectDataset

Name	Description	Value
type	Type of dataset.	'GoogleBigQueryObject' (required)

GreenplumTableDataset

Name	Description	Value
type	Type of dataset.	'GreenplumTable' (required)

HBaseObjectDataset

Name	Description	Value
type	Type of dataset.	'HBaseObject' (required)

HiveObjectDataset

Name	Description	Value
type	Type of dataset.	'HiveObject' (required)

HttpDataset

Name	Description	Value
type	Type of dataset.	'HttpFile' (required)
typeProperties	Properties specific to this dataset type.	HttpDatasetTypeProperties (required)

HttpDatasetTypeProperties

Name	Description	Value
additionalHeaders	The headers for the HTTP Request. e.g. request-header-name-1:request-header-value-1 ... request-header-name-n:request-header-value-n Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.
compression	The data compression method used on files.	DatasetCompression
format	The format of files.	DatasetStorageFormat
relativeUrl	The relative URL based on the URL in the HttpLinkedService refers to an HTTP file Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.
requestBody	The body for the HTTP request. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.
requestMethod	The HTTP method for the HTTP request. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.

HubspotObjectDataset

Name	Description	Value
type	Type of dataset.	'HubspotObject' (required)

ImpalaObjectDataset

Name	Description	Value
type	Type of dataset.	'ImpalaObject' (required)

JiraObjectDataset

Name	Description	Value
type	Type of dataset.	'JiraObject' (required)

MagentoObjectDataset

Name	Description	Value
type	Type of dataset.	'MagentoObject' (required)

MariaDBTableDataset

Name	Description	Value
type	Type of dataset.	'MariaDBTable' (required)

MarketoObjectDataset

Name	Description	Value
type	Type of dataset.	'MarketoObject' (required)

MongoDbCollectionDataset

Name	Description	Value
type	Type of dataset.	'MongoDbCollection' (required)
typeProperties	MongoDB database dataset properties.	MongoDbCollectionDatasetTypeProperties (required)

MongoDbCollectionDatasetTypeProperties

Name	Description	Value
collectionName	The table name of the MongoDB database. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.(required)

NetezzaTableDataset

Name	Description	Value
type	Type of dataset.	'NetezzaTable' (required)

ODataResourceDataset

Name	Description	Value
type	Type of dataset.	'ODataResource' (required)
typeProperties	OData dataset properties.	ODataResourceDatasetTypeProperties (required)

ODataResourceDatasetTypeProperties

Name	Description	Value
path	The OData resource path. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.

OracleTableDataset

Name	Description	Value
type	Type of dataset.	'OracleTable' (required)
typeProperties	On-premises Oracle dataset properties.	OracleTableDatasetTypeProperties (required)

OracleTableDatasetTypeProperties

Name	Description	Value
tableName	The table name of the on-premises Oracle database. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.(required)

PaypalObjectDataset

Name	Description	Value
type	Type of dataset.	'PaypalObject' (required)

PhoenixObjectDataset

Name	Description	Value
type	Type of dataset.	'PhoenixObject' (required)

PrestoObjectDataset

Name	Description	Value
type	Type of dataset.	'PrestoObject' (required)

QuickBooksObjectDataset

Name	Description	Value
type	Type of dataset.	'QuickBooksObject' (required)

RelationalTableDataset

Name	Description	Value
type	Type of dataset.	'RelationalTable' (required)
typeProperties	Relational table dataset properties.	RelationalTableDatasetTypeProperties (required)

RelationalTableDatasetTypeProperties

Name	Description	Value
tableName	The relational table name. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.

ResponsysObjectDataset

Name	Description	Value
type	Type of dataset.	'ResponsysObject' (required)

SalesforceMarketingCloudObjectDataset

Name	Description	Value
type	Type of dataset.	'SalesforceMarketingCloudObject' (required)

SalesforceObjectDataset

Name	Description	Value
type	Type of dataset.	'SalesforceObject' (required)
typeProperties	Salesforce object dataset properties.	SalesforceObjectDatasetTypeProperties (required)

SalesforceObjectDatasetTypeProperties

Name	Description	Value
objectApiName	The Salesforce object API name. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.

SapCloudForCustomerResourceDataset

Name	Description	Value
type	Type of dataset.	'SapCloudForCustomerResource' (required)
typeProperties	SAP Cloud For Customer OData resource dataset properties.	SapCloudForCustomerResourceDatasetTypeProperties (required)

SapCloudForCustomerResourceDatasetTypeProperties

Name	Description	Value
path	The path of the SAP Cloud for Customer OData entity. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.(required)

SapEccResourceDataset

Name	Description	Value
type	Type of dataset.	'SapEccResource' (required)
typeProperties	SAP ECC OData resource dataset properties.	SapEccResourceDatasetTypeProperties (required)

SapEccResourceDatasetTypeProperties

Name	Description	Value
path	The path of the SAP ECC OData entity. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.(required)

ServiceNowObjectDataset

Name	Description	Value
type	Type of dataset.	'ServiceNowObject' (required)

ShopifyObjectDataset

Name	Description	Value
type	Type of dataset.	'ShopifyObject' (required)

SparkObjectDataset

Name	Description	Value
type	Type of dataset.	'SparkObject' (required)

SqlServerTableDataset

Name	Description	Value
type	Type of dataset.	'SqlServerTable' (required)
typeProperties	On-premises SQL Server dataset properties.	SqlServerTableDatasetTypeProperties (required)

SqlServerTableDatasetTypeProperties

Name	Description	Value
tableName	The table name of the SQL Server dataset. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.(required)

SquareObjectDataset

Name	Description	Value
type	Type of dataset.	'SquareObject' (required)

VerticaTableDataset

Name	Description	Value
type	Type of dataset.	'VerticaTable' (required)

WebTableDataset

Name	Description	Value
type	Type of dataset.	'WebTable' (required)
typeProperties	Web table dataset properties.	WebTableDatasetTypeProperties (required)

WebTableDatasetTypeProperties

Name	Description	Value
index	The zero-based index of the table in the web page. Type: integer (or Expression with resultType integer), minimum: 0.	For Bicep, you can use the any() function.(required)
path	The relative URL to the web page from the linked service URL. Type: string (or Expression with resultType string).	For Bicep, you can use the any() function.

XeroObjectDataset

Name	Description	Value
type	Type of dataset.	'XeroObject' (required)

ZohoObjectDataset

Name	Description	Value
type	Type of dataset.	'ZohoObject' (required)

Quickstart templates

The following quickstart templates deploy this resource type.

Template	Description
More is possible with Azure Data Factory - One click to try Azure Data Factory	This template creates a data factory pipeline for a copy activity from Azure Blob into another Azure Blob
Create a Data Factory, copy from Salesforce to Blob	This template uses Azure Data Factory to perform data movement from Salesforce to Azure Blobs
Bulk copy using Azure Data Factory	This template creates a V2 data factory that copies data in bulk an Azure SQL database to an Azure SQL data warehouse.
Create a V2 data factory	This template creates a V2 data factory that copies data from a folder in an Azure Blob Storage to another folder in the storage.
Create a V2 data factory (MySQL)	This template creates a V2 data factory that copies data from a folder in an Azure Blob Storage to a table in an Azure Database for MySQL.
Create a V2 data factory (PostGre)	This template creates a V2 data factory that copies data from a folder in an Azure Blob Storage to a table in an Azure Database for PostgreSQL.
Create a V2 data factory (SQL)	This template creates a V2 data factory that copies data from a folder in an Azure Blob Storage to a table in an Azure SQL database.
Create a V2 data factory (SQL On-prem)	This template creates a V2 data factory that copies data from an on-premises SQL Server to an Azure blob storage

ARM template resource definition

The factories/datasets resource type can be deployed with operations that target:

Resource groups - See resource group deployment commands

For a list of changed properties in each API version, see change log.

Resource format

To create a Microsoft.DataFactory/factories/datasets resource, add the following JSON to your template.

{
  "type": "Microsoft.DataFactory/factories/datasets",
  "apiVersion": "2017-09-01-preview",
  "name": "string",
  "properties": {
    "annotations": [ object ],
    "description": "string",
    "linkedServiceName": {
      "parameters": {
        "{customized property}": {}
      },
      "referenceName": "string",
      "type": "LinkedServiceReference"
    },
    "parameters": {
      "{customized property}": {
        "defaultValue": {},
        "type": "string"
      }
    },
    "structure": {},
    "type": "string"
    // For remaining properties, see Dataset objects
  }
}

Dataset objects

Set the type property to specify the type of object.

For AmazonMWSObject, use:

  "type": "AmazonMWSObject"

For AmazonS3Object, use:

  "type": "AmazonS3Object",
  "typeProperties": {
    "bucketName": {},
    "compression": {
      "type": "string"
      // For remaining properties, see DatasetCompression objects
    },
    "format": {
      "deserializer": {},
      "serializer": {}
      // For remaining properties, see DatasetStorageFormat objects
    },
    "key": {},
    "prefix": {},
    "version": {}
  }

For AzureBlob, use:

  "type": "AzureBlob",
  "typeProperties": {
    "compression": {
      "type": "string"
      // For remaining properties, see DatasetCompression objects
    },
    "fileName": {},
    "folderPath": {},
    "format": {
      "deserializer": {},
      "serializer": {}
      // For remaining properties, see DatasetStorageFormat objects
    },
    "tableRootLocation": {}
  }

For AzureDataLakeStoreFile, use:

  "type": "AzureDataLakeStoreFile",
  "typeProperties": {
    "compression": {
      "type": "string"
      // For remaining properties, see DatasetCompression objects
    },
    "fileName": {},
    "folderPath": {},
    "format": {
      "deserializer": {},
      "serializer": {}
      // For remaining properties, see DatasetStorageFormat objects
    }
  }

For AzureMySqlTable, use:

  "type": "AzureMySqlTable",
  "typeProperties": {
    "tableName": {}
  }

For AzurePostgreSqlTable, use:

  "type": "AzurePostgreSqlTable"

For AzureSearchIndex, use:

  "type": "AzureSearchIndex",
  "typeProperties": {
    "indexName": {}
  }

For AzureSqlDWTable, use:

  "type": "AzureSqlDWTable",
  "typeProperties": {
    "tableName": {}
  }

For AzureSqlTable, use:

  "type": "AzureSqlTable",
  "typeProperties": {
    "tableName": {}
  }

For AzureTable, use:

  "type": "AzureTable",
  "typeProperties": {
    "tableName": {}
  }

For CassandraTable, use:

  "type": "CassandraTable",
  "typeProperties": {
    "keyspace": {},
    "tableName": {}
  }

For ConcurObject, use:

  "type": "ConcurObject"

For CouchbaseTable, use:

  "type": "CouchbaseTable"

For DocumentDbCollection, use:

  "type": "DocumentDbCollection",
  "typeProperties": {
    "collectionName": {}
  }

For DrillTable, use:

  "type": "DrillTable"

For DynamicsEntity, use:

  "type": "DynamicsEntity",
  "typeProperties": {
    "entityName": {}
  }

For EloquaObject, use:

  "type": "EloquaObject"

For FileShare, use:

  "type": "FileShare",
  "typeProperties": {
    "compression": {
      "type": "string"
      // For remaining properties, see DatasetCompression objects
    },
    "fileFilter": {},
    "fileName": {},
    "folderPath": {},
    "format": {
      "deserializer": {},
      "serializer": {}
      // For remaining properties, see DatasetStorageFormat objects
    }
  }

For GoogleBigQueryObject, use:

  "type": "GoogleBigQueryObject"

For GreenplumTable, use:

  "type": "GreenplumTable"

For HBaseObject, use:

  "type": "HBaseObject"

For HiveObject, use:

  "type": "HiveObject"

For HttpFile, use:

  "type": "HttpFile",
  "typeProperties": {
    "additionalHeaders": {},
    "compression": {
      "type": "string"
      // For remaining properties, see DatasetCompression objects
    },
    "format": {
      "deserializer": {},
      "serializer": {}
      // For remaining properties, see DatasetStorageFormat objects
    },
    "relativeUrl": {},
    "requestBody": {},
    "requestMethod": {}
  }

For HubspotObject, use:

  "type": "HubspotObject"

For ImpalaObject, use:

  "type": "ImpalaObject"

For JiraObject, use:

  "type": "JiraObject"

For MagentoObject, use:

  "type": "MagentoObject"

For MariaDBTable, use:

  "type": "MariaDBTable"

For MarketoObject, use:

  "type": "MarketoObject"

For MongoDbCollection, use:

  "type": "MongoDbCollection",
  "typeProperties": {
    "collectionName": {}
  }

For NetezzaTable, use:

  "type": "NetezzaTable"

For ODataResource, use:

  "type": "ODataResource",
  "typeProperties": {
    "path": {}
  }

For OracleTable, use:

  "type": "OracleTable",
  "typeProperties": {
    "tableName": {}
  }

For PaypalObject, use:

  "type": "PaypalObject"

For PhoenixObject, use:

  "type": "PhoenixObject"

For PrestoObject, use:

  "type": "PrestoObject"

For QuickBooksObject, use:

  "type": "QuickBooksObject"

For RelationalTable, use:

  "type": "RelationalTable",
  "typeProperties": {
    "tableName": {}
  }

For ResponsysObject, use:

  "type": "ResponsysObject"

For SalesforceMarketingCloudObject, use:

  "type": "SalesforceMarketingCloudObject"

For SalesforceObject, use:

  "type": "SalesforceObject",
  "typeProperties": {
    "objectApiName": {}
  }

For SapCloudForCustomerResource, use:

  "type": "SapCloudForCustomerResource",
  "typeProperties": {
    "path": {}
  }

For SapEccResource, use:

  "type": "SapEccResource",
  "typeProperties": {
    "path": {}
  }

For ServiceNowObject, use:

  "type": "ServiceNowObject"

For ShopifyObject, use:

  "type": "ShopifyObject"

For SparkObject, use:

  "type": "SparkObject"

For SqlServerTable, use:

  "type": "SqlServerTable",
  "typeProperties": {
    "tableName": {}
  }

For SquareObject, use:

  "type": "SquareObject"

For VerticaTable, use:

  "type": "VerticaTable"

For WebTable, use:

  "type": "WebTable",
  "typeProperties": {
    "index": {},
    "path": {}
  }

For XeroObject, use:

  "type": "XeroObject"

For ZohoObject, use:

  "type": "ZohoObject"

DatasetCompression objects

Set the type property to specify the type of object.

For BZip2, use:

  "type": "BZip2"

For Deflate, use:

  "type": "Deflate",
  "level": "string"

For GZip, use:

  "type": "GZip",
  "level": "string"

For ZipDeflate, use:

  "type": "ZipDeflate",
  "level": "string"

DatasetStorageFormat objects

Set the type property to specify the type of object.

Property values

factories/datasets

Name	Description	Value
type	The resource type	'Microsoft.DataFactory/factories/datasets'
apiVersion	The resource api version	'2017-09-01-preview'
name	The resource name See how to set names and types for child resources in JSON ARM templates.	string (required) Character limit: 1-260 Valid characters: Can't use: `<>*#.%&:\\+?/-` or control characters Start with alphanumeric.
properties	Dataset properties.	Dataset (required)

Dataset

Name	Description	Value
annotations	List of tags that can be used for describing the Dataset.	any[]
description	Dataset description.	string
linkedServiceName	Linked service reference.	LinkedServiceReference (required)
parameters	Parameters for dataset.	ParameterDefinitionSpecification
structure	Columns that define the structure of the dataset. Type: array (or Expression with resultType array), itemType: DatasetDataElement.
type	Set the object type	AmazonMWSObject AmazonS3Object AzureBlob AzureDataLakeStoreFile AzureMySqlTable AzurePostgreSqlTable AzureSearchIndex AzureSqlDWTable AzureSqlTable AzureTable CassandraTable ConcurObject CouchbaseTable DocumentDbCollection DrillTable DynamicsEntity EloquaObject FileShare GoogleBigQueryObject GreenplumTable HBaseObject HiveObject HttpFile HubspotObject ImpalaObject JiraObject MagentoObject MariaDBTable MarketoObject MongoDbCollection NetezzaTable ODataResource OracleTable PaypalObject PhoenixObject PrestoObject QuickBooksObject RelationalTable ResponsysObject SalesforceMarketingCloudObject SalesforceObject SapCloudForCustomerResource SapEccResource ServiceNowObject ShopifyObject SparkObject SqlServerTable SquareObject VerticaTable WebTable XeroObject ZohoObject (required)

LinkedServiceReference

Name	Description	Value
parameters	Arguments for LinkedService.	ParameterValueSpecification
referenceName	Reference LinkedService name.	string (required)
type	Linked service reference type.	'LinkedServiceReference' (required)

ParameterValueSpecification

Name	Description	Value
{customized property}

ParameterDefinitionSpecification

Name	Description	Value
{customized property}		ParameterSpecification

ParameterSpecification

Name	Description	Value
defaultValue	Default value of parameter.
type	Parameter type.	'Array' 'Bool' 'Float' 'Int' 'Object' 'SecureString' 'String' (required)

AmazonMWSObjectDataset

Name	Description	Value
type	Type of dataset.	'AmazonMWSObject' (required)

AmazonS3Dataset

Name	Description	Value
type	Type of dataset.	'AmazonS3Object' (required)
typeProperties	Amazon S3 dataset properties.	AmazonS3DatasetTypeProperties (required)

AmazonS3DatasetTypeProperties

Name	Description	Value
bucketName	The name of the Amazon S3 bucket. Type: string (or Expression with resultType string).
compression	The data compression method used for the Amazon S3 object.	DatasetCompression
format	The format of files.	DatasetStorageFormat
key	The key of the Amazon S3 object. Type: string (or Expression with resultType string).
prefix	The prefix filter for the S3 object name. Type: string (or Expression with resultType string).
version	The version for the S3 object. Type: string (or Expression with resultType string).

DatasetCompression

Name	Description	Value
type	Set the object type	BZip2 Deflate GZip ZipDeflate (required)

DatasetBZip2Compression

Name	Description	Value
type	Type of dataset compression.	'BZip2' (required)

DatasetDeflateCompression

Name	Description	Value
type	Type of dataset compression.	'Deflate' (required)
level	The Deflate compression level.	'Fastest' 'Optimal'

DatasetGZipCompression

Name	Description	Value
type	Type of dataset compression.	'GZip' (required)
level	The GZip compression level.	'Fastest' 'Optimal'

DatasetZipDeflateCompression

Name	Description	Value
type	Type of dataset compression.	'ZipDeflate' (required)
level	The ZipDeflate compression level.	'Fastest' 'Optimal'

DatasetStorageFormat

Name	Description	Value
deserializer	Deserializer. Type: string (or Expression with resultType string).
serializer	Serializer. Type: string (or Expression with resultType string).

AzureBlobDataset

Name	Description	Value
type	Type of dataset.	'AzureBlob' (required)
typeProperties	Azure Blob dataset properties.	AzureBlobDatasetTypeProperties (required)

AzureBlobDatasetTypeProperties

Name	Description	Value
compression	The data compression method used for the blob storage.	DatasetCompression
fileName	The name of the Azure Blob. Type: string (or Expression with resultType string).
folderPath	The path of the Azure Blob storage. Type: string (or Expression with resultType string).
format	The format of the Azure Blob storage.	DatasetStorageFormat
tableRootLocation	The root of blob path. Type: string (or Expression with resultType string).

AzureDataLakeStoreDataset

Name	Description	Value
type	Type of dataset.	'AzureDataLakeStoreFile' (required)
typeProperties	Azure Data Lake Store dataset properties.	AzureDataLakeStoreDatasetTypeProperties (required)

AzureDataLakeStoreDatasetTypeProperties

Name	Description	Value
compression	The data compression method used for the item(s) in the Azure Data Lake Store.	DatasetCompression
fileName	The name of the file in the Azure Data Lake Store. Type: string (or Expression with resultType string).
folderPath	Path to the folder in the Azure Data Lake Store. Type: string (or Expression with resultType string).
format	The format of the Data Lake Store.	DatasetStorageFormat

AzureMySqlTableDataset

Name	Description	Value
type	Type of dataset.	'AzureMySqlTable' (required)
typeProperties	Azure MySQL database dataset properties.	AzureMySqlTableDatasetTypeProperties (required)

AzureMySqlTableDatasetTypeProperties

Name	Description	Value
tableName	The Azure MySQL database table name. Type: string (or Expression with resultType string).

AzurePostgreSqlTableDataset

Name	Description	Value
type	Type of dataset.	'AzurePostgreSqlTable' (required)

AzureSearchIndexDataset

Name	Description	Value
type	Type of dataset.	'AzureSearchIndex' (required)
typeProperties	Properties specific to this dataset type.	AzureSearchIndexDatasetTypeProperties (required)

AzureSearchIndexDatasetTypeProperties

Name	Description	Value
indexName	The name of the Azure Search Index. Type: string (or Expression with resultType string).

AzureSqlDWTableDataset

Name	Description	Value
type	Type of dataset.	'AzureSqlDWTable' (required)
typeProperties	Azure SQL Data Warehouse dataset properties.	AzureSqlDWTableDatasetTypeProperties (required)

AzureSqlDWTableDatasetTypeProperties

Name	Description	Value
tableName	The table name of the Azure SQL Data Warehouse. Type: string (or Expression with resultType string).

AzureSqlTableDataset

Name	Description	Value
type	Type of dataset.	'AzureSqlTable' (required)
typeProperties	Azure SQL dataset properties.	AzureSqlTableDatasetTypeProperties (required)

AzureSqlTableDatasetTypeProperties

Name	Description	Value
tableName	The table name of the Azure SQL database. Type: string (or Expression with resultType string).

AzureTableDataset

Name	Description	Value
type	Type of dataset.	'AzureTable' (required)
typeProperties	Azure Table dataset properties.	AzureTableDatasetTypeProperties (required)

AzureTableDatasetTypeProperties

Name	Description	Value
tableName	The table name of the Azure Table storage. Type: string (or Expression with resultType string).

CassandraTableDataset

Name	Description	Value
type	Type of dataset.	'CassandraTable' (required)
typeProperties	Cassandra dataset properties.	CassandraTableDatasetTypeProperties (required)

CassandraTableDatasetTypeProperties

Name	Description	Value
keyspace	The keyspace of the Cassandra database. Type: string (or Expression with resultType string).
tableName	The table name of the Cassandra database. Type: string (or Expression with resultType string).

ConcurObjectDataset

Name	Description	Value
type	Type of dataset.	'ConcurObject' (required)

CouchbaseTableDataset

Name	Description	Value
type	Type of dataset.	'CouchbaseTable' (required)

DocumentDbCollectionDataset

Name	Description	Value
type	Type of dataset.	'DocumentDbCollection' (required)
typeProperties	DocumentDB Collection dataset properties.	DocumentDbCollectionDatasetTypeProperties (required)

DocumentDbCollectionDatasetTypeProperties

Name	Description	Value
collectionName	Document Database collection name. Type: string (or Expression with resultType string).

DrillTableDataset

Name	Description	Value
type	Type of dataset.	'DrillTable' (required)

DynamicsEntityDataset

Name	Description	Value
type	Type of dataset.	'DynamicsEntity' (required)
typeProperties	Dynamics entity dataset properties.	DynamicsEntityDatasetTypeProperties (required)

DynamicsEntityDatasetTypeProperties

Name	Description	Value
entityName	The logical name of the entity. Type: string (or Expression with resultType string).

EloquaObjectDataset

Name	Description	Value
type	Type of dataset.	'EloquaObject' (required)

FileShareDataset

Name	Description	Value
type	Type of dataset.	'FileShare' (required)
typeProperties	On-premises file system dataset properties.	FileShareDatasetTypeProperties (required)

FileShareDatasetTypeProperties

Name	Description	Value
compression	The data compression method used for the file system.	DatasetCompression
fileFilter	Specify a filter to be used to select a subset of files in the folderPath rather than all files. Type: string (or Expression with resultType string).
fileName	The name of the on-premises file system. Type: string (or Expression with resultType string).
folderPath	The path of the on-premises file system. Type: string (or Expression with resultType string).
format	The format of the files.	DatasetStorageFormat

GoogleBigQueryObjectDataset

Name	Description	Value
type	Type of dataset.	'GoogleBigQueryObject' (required)

GreenplumTableDataset

Name	Description	Value
type	Type of dataset.	'GreenplumTable' (required)

HBaseObjectDataset

Name	Description	Value
type	Type of dataset.	'HBaseObject' (required)

HiveObjectDataset

Name	Description	Value
type	Type of dataset.	'HiveObject' (required)

HttpDataset

Name	Description	Value
type	Type of dataset.	'HttpFile' (required)
typeProperties	Properties specific to this dataset type.	HttpDatasetTypeProperties (required)

HttpDatasetTypeProperties

Name	Description	Value
additionalHeaders	The headers for the HTTP Request. e.g. request-header-name-1:request-header-value-1 ... request-header-name-n:request-header-value-n Type: string (or Expression with resultType string).
compression	The data compression method used on files.	DatasetCompression
format	The format of files.	DatasetStorageFormat
relativeUrl	The relative URL based on the URL in the HttpLinkedService refers to an HTTP file Type: string (or Expression with resultType string).
requestBody	The body for the HTTP request. Type: string (or Expression with resultType string).
requestMethod	The HTTP method for the HTTP request. Type: string (or Expression with resultType string).

HubspotObjectDataset

Name	Description	Value
type	Type of dataset.	'HubspotObject' (required)

ImpalaObjectDataset

Name	Description	Value
type	Type of dataset.	'ImpalaObject' (required)

JiraObjectDataset

Name	Description	Value
type	Type of dataset.	'JiraObject' (required)

MagentoObjectDataset

Name	Description	Value
type	Type of dataset.	'MagentoObject' (required)

MariaDBTableDataset

Name	Description	Value
type	Type of dataset.	'MariaDBTable' (required)

MarketoObjectDataset

Name	Description	Value
type	Type of dataset.	'MarketoObject' (required)

MongoDbCollectionDataset

Name	Description	Value
type	Type of dataset.	'MongoDbCollection' (required)
typeProperties	MongoDB database dataset properties.	MongoDbCollectionDatasetTypeProperties (required)

MongoDbCollectionDatasetTypeProperties

Name	Description	Value
collectionName	The table name of the MongoDB database. Type: string (or Expression with resultType string).

NetezzaTableDataset

Name	Description	Value
type	Type of dataset.	'NetezzaTable' (required)

ODataResourceDataset

Name	Description	Value
type	Type of dataset.	'ODataResource' (required)
typeProperties	OData dataset properties.	ODataResourceDatasetTypeProperties (required)

ODataResourceDatasetTypeProperties

Name	Description	Value
path	The OData resource path. Type: string (or Expression with resultType string).

OracleTableDataset

Name	Description	Value
type	Type of dataset.	'OracleTable' (required)
typeProperties	On-premises Oracle dataset properties.	OracleTableDatasetTypeProperties (required)

OracleTableDatasetTypeProperties

Name	Description	Value
tableName	The table name of the on-premises Oracle database. Type: string (or Expression with resultType string).

PaypalObjectDataset

Name	Description	Value
type	Type of dataset.	'PaypalObject' (required)

PhoenixObjectDataset

Name	Description	Value
type	Type of dataset.	'PhoenixObject' (required)

PrestoObjectDataset

Name	Description	Value
type	Type of dataset.	'PrestoObject' (required)

QuickBooksObjectDataset

Name	Description	Value
type	Type of dataset.	'QuickBooksObject' (required)

RelationalTableDataset

Name	Description	Value
type	Type of dataset.	'RelationalTable' (required)
typeProperties	Relational table dataset properties.	RelationalTableDatasetTypeProperties (required)

RelationalTableDatasetTypeProperties

Name	Description	Value
tableName	The relational table name. Type: string (or Expression with resultType string).

ResponsysObjectDataset

Name	Description	Value
type	Type of dataset.	'ResponsysObject' (required)

SalesforceMarketingCloudObjectDataset

Name	Description	Value
type	Type of dataset.	'SalesforceMarketingCloudObject' (required)

SalesforceObjectDataset

Name	Description	Value
type	Type of dataset.	'SalesforceObject' (required)
typeProperties	Salesforce object dataset properties.	SalesforceObjectDatasetTypeProperties (required)

SalesforceObjectDatasetTypeProperties

Name	Description	Value
objectApiName	The Salesforce object API name. Type: string (or Expression with resultType string).

SapCloudForCustomerResourceDataset

Name	Description	Value
type	Type of dataset.	'SapCloudForCustomerResource' (required)
typeProperties	SAP Cloud For Customer OData resource dataset properties.	SapCloudForCustomerResourceDatasetTypeProperties (required)

SapCloudForCustomerResourceDatasetTypeProperties

Name	Description	Value
path	The path of the SAP Cloud for Customer OData entity. Type: string (or Expression with resultType string).

SapEccResourceDataset

Name	Description	Value
type	Type of dataset.	'SapEccResource' (required)
typeProperties	SAP ECC OData resource dataset properties.	SapEccResourceDatasetTypeProperties (required)

SapEccResourceDatasetTypeProperties

Name	Description	Value
path	The path of the SAP ECC OData entity. Type: string (or Expression with resultType string).

ServiceNowObjectDataset

Name	Description	Value
type	Type of dataset.	'ServiceNowObject' (required)

ShopifyObjectDataset

Name	Description	Value
type	Type of dataset.	'ShopifyObject' (required)

SparkObjectDataset

Name	Description	Value
type	Type of dataset.	'SparkObject' (required)

SqlServerTableDataset

Name	Description	Value
type	Type of dataset.	'SqlServerTable' (required)
typeProperties	On-premises SQL Server dataset properties.	SqlServerTableDatasetTypeProperties (required)

SqlServerTableDatasetTypeProperties

Name	Description	Value
tableName	The table name of the SQL Server dataset. Type: string (or Expression with resultType string).

SquareObjectDataset

Name	Description	Value
type	Type of dataset.	'SquareObject' (required)

VerticaTableDataset

Name	Description	Value
type	Type of dataset.	'VerticaTable' (required)

WebTableDataset

Name	Description	Value
type	Type of dataset.	'WebTable' (required)
typeProperties	Web table dataset properties.	WebTableDatasetTypeProperties (required)

WebTableDatasetTypeProperties

Name	Description	Value
index	The zero-based index of the table in the web page. Type: integer (or Expression with resultType integer), minimum: 0.
path	The relative URL to the web page from the linked service URL. Type: string (or Expression with resultType string).

XeroObjectDataset

Name	Description	Value
type	Type of dataset.	'XeroObject' (required)

ZohoObjectDataset

Name	Description	Value
type	Type of dataset.	'ZohoObject' (required)

Quickstart templates

The following quickstart templates deploy this resource type.

Template	Description
More is possible with Azure Data Factory - One click to try Azure Data Factory	This template creates a data factory pipeline for a copy activity from Azure Blob into another Azure Blob
Create a Data Factory, copy from Salesforce to Blob	This template uses Azure Data Factory to perform data movement from Salesforce to Azure Blobs
Bulk copy using Azure Data Factory	This template creates a V2 data factory that copies data in bulk an Azure SQL database to an Azure SQL data warehouse.
Create a V2 data factory	This template creates a V2 data factory that copies data from a folder in an Azure Blob Storage to another folder in the storage.
Create a V2 data factory (MySQL)	This template creates a V2 data factory that copies data from a folder in an Azure Blob Storage to a table in an Azure Database for MySQL.
Create a V2 data factory (PostGre)	This template creates a V2 data factory that copies data from a folder in an Azure Blob Storage to a table in an Azure Database for PostgreSQL.
Create a V2 data factory (SQL)	This template creates a V2 data factory that copies data from a folder in an Azure Blob Storage to a table in an Azure SQL database.
Create a V2 data factory (SQL On-prem)	This template creates a V2 data factory that copies data from an on-premises SQL Server to an Azure blob storage

Terraform (AzAPI provider) resource definition

The factories/datasets resource type can be deployed with operations that target:

Resource groups

For a list of changed properties in each API version, see change log.

Resource format

To create a Microsoft.DataFactory/factories/datasets resource, add the following Terraform to your template.

resource "azapi_resource" "symbolicname" {
  type = "Microsoft.DataFactory/factories/datasets@2017-09-01-preview"
  name = "string"
  parent_id = "string"
  body = jsonencode({
    properties = {
      annotations = [ object ]
      description = "string"
      linkedServiceName = {
        parameters = {}
        referenceName = "string"
        type = "LinkedServiceReference"
      }
      parameters = {
        {customized property} = {
          type = "string"
        }
      }
      type = "string"
      // For remaining properties, see Dataset objects
    }
  })
}

Dataset objects

Set the type property to specify the type of object.

For AmazonMWSObject, use:

  type = "AmazonMWSObject"

For AmazonS3Object, use:

  type = "AmazonS3Object"
  typeProperties = {
    compression = {
      type = "string"
      // For remaining properties, see DatasetCompression objects
    }
    format = {
      // For remaining properties, see DatasetStorageFormat objects
    }
  }

For AzureBlob, use:

  type = "AzureBlob"
  typeProperties = {
    compression = {
      type = "string"
      // For remaining properties, see DatasetCompression objects
    }
    format = {
      // For remaining properties, see DatasetStorageFormat objects
    }
  }

For AzureDataLakeStoreFile, use:

  type = "AzureDataLakeStoreFile"
  typeProperties = {
    compression = {
      type = "string"
      // For remaining properties, see DatasetCompression objects
    }
    format = {
      // For remaining properties, see DatasetStorageFormat objects
    }
  }

For AzureMySqlTable, use:

  type = "AzureMySqlTable"
  typeProperties = {}

For AzurePostgreSqlTable, use:

  type = "AzurePostgreSqlTable"

For AzureSearchIndex, use:

  type = "AzureSearchIndex"
  typeProperties = {}

For AzureSqlDWTable, use:

  type = "AzureSqlDWTable"
  typeProperties = {}

For AzureSqlTable, use:

  type = "AzureSqlTable"
  typeProperties = {}

For AzureTable, use:

  type = "AzureTable"
  typeProperties = {}

For CassandraTable, use:

  type = "CassandraTable"
  typeProperties = {}

For ConcurObject, use:

  type = "ConcurObject"

For CouchbaseTable, use:

  type = "CouchbaseTable"

For DocumentDbCollection, use:

  type = "DocumentDbCollection"
  typeProperties = {}

For DrillTable, use:

  type = "DrillTable"

For DynamicsEntity, use:

  type = "DynamicsEntity"
  typeProperties = {}

For EloquaObject, use:

  type = "EloquaObject"

For FileShare, use:

  type = "FileShare"
  typeProperties = {
    compression = {
      type = "string"
      // For remaining properties, see DatasetCompression objects
    }
    format = {
      // For remaining properties, see DatasetStorageFormat objects
    }
  }

For GoogleBigQueryObject, use:

  type = "GoogleBigQueryObject"

For GreenplumTable, use:

  type = "GreenplumTable"

For HBaseObject, use:

  type = "HBaseObject"

For HiveObject, use:

  type = "HiveObject"

For HttpFile, use:

  type = "HttpFile"
  typeProperties = {
    compression = {
      type = "string"
      // For remaining properties, see DatasetCompression objects
    }
    format = {
      // For remaining properties, see DatasetStorageFormat objects
    }
  }

For HubspotObject, use:

  type = "HubspotObject"

For ImpalaObject, use:

  type = "ImpalaObject"

For JiraObject, use:

  type = "JiraObject"

For MagentoObject, use:

  type = "MagentoObject"

For MariaDBTable, use:

  type = "MariaDBTable"

For MarketoObject, use:

  type = "MarketoObject"

For MongoDbCollection, use:

  type = "MongoDbCollection"
  typeProperties = {}

For NetezzaTable, use:

  type = "NetezzaTable"

For ODataResource, use:

  type = "ODataResource"
  typeProperties = {}

For OracleTable, use:

  type = "OracleTable"
  typeProperties = {}

For PaypalObject, use:

  type = "PaypalObject"

For PhoenixObject, use:

  type = "PhoenixObject"

For PrestoObject, use:

  type = "PrestoObject"

For QuickBooksObject, use:

  type = "QuickBooksObject"

For RelationalTable, use:

  type = "RelationalTable"
  typeProperties = {}

For ResponsysObject, use:

  type = "ResponsysObject"

For SalesforceMarketingCloudObject, use:

  type = "SalesforceMarketingCloudObject"

For SalesforceObject, use:

  type = "SalesforceObject"
  typeProperties = {}

For SapCloudForCustomerResource, use:

  type = "SapCloudForCustomerResource"
  typeProperties = {}

For SapEccResource, use:

  type = "SapEccResource"
  typeProperties = {}

For ServiceNowObject, use:

  type = "ServiceNowObject"

For ShopifyObject, use:

  type = "ShopifyObject"

For SparkObject, use:

  type = "SparkObject"

For SqlServerTable, use:

  type = "SqlServerTable"
  typeProperties = {}

For SquareObject, use:

  type = "SquareObject"

For VerticaTable, use:

  type = "VerticaTable"

For WebTable, use:

  type = "WebTable"
  typeProperties = {}

For XeroObject, use:

  type = "XeroObject"

For ZohoObject, use:

  type = "ZohoObject"

DatasetCompression objects

Set the type property to specify the type of object.

For BZip2, use:

  type = "BZip2"

For Deflate, use:

  type = "Deflate"
  level = "string"

For GZip, use:

  type = "GZip"
  level = "string"

For ZipDeflate, use:

  type = "ZipDeflate"
  level = "string"

DatasetStorageFormat objects

Set the type property to specify the type of object.

Property values

factories/datasets

Name	Description	Value
type	The resource type	"Microsoft.DataFactory/factories/datasets@2017-09-01-preview"
name	The resource name	string (required) Character limit: 1-260 Valid characters: Can't use: `<>*#.%&:\\+?/-` or control characters Start with alphanumeric.
parent_id	The ID of the resource that is the parent for this resource.	ID for resource of type: factories
properties	Dataset properties.	Dataset (required)

Dataset

Name	Description	Value
annotations	List of tags that can be used for describing the Dataset.	any[]
description	Dataset description.	string
linkedServiceName	Linked service reference.	LinkedServiceReference (required)
parameters	Parameters for dataset.	ParameterDefinitionSpecification
structure	Columns that define the structure of the dataset. Type: array (or Expression with resultType array), itemType: DatasetDataElement.
type	Set the object type	AmazonMWSObject AmazonS3Object AzureBlob AzureDataLakeStoreFile AzureMySqlTable AzurePostgreSqlTable AzureSearchIndex AzureSqlDWTable AzureSqlTable AzureTable CassandraTable ConcurObject CouchbaseTable DocumentDbCollection DrillTable DynamicsEntity EloquaObject FileShare GoogleBigQueryObject GreenplumTable HBaseObject HiveObject HttpFile HubspotObject ImpalaObject JiraObject MagentoObject MariaDBTable MarketoObject MongoDbCollection NetezzaTable ODataResource OracleTable PaypalObject PhoenixObject PrestoObject QuickBooksObject RelationalTable ResponsysObject SalesforceMarketingCloudObject SalesforceObject SapCloudForCustomerResource SapEccResource ServiceNowObject ShopifyObject SparkObject SqlServerTable SquareObject VerticaTable WebTable XeroObject ZohoObject (required)

LinkedServiceReference

Name	Description	Value
parameters	Arguments for LinkedService.	ParameterValueSpecification
referenceName	Reference LinkedService name.	string (required)
type	Linked service reference type.	"LinkedServiceReference" (required)

ParameterValueSpecification

Name	Description	Value
{customized property}

ParameterDefinitionSpecification

Name	Description	Value
{customized property}		ParameterSpecification

ParameterSpecification

Name	Description	Value
defaultValue	Default value of parameter.
type	Parameter type.	"Array" "Bool" "Float" "Int" "Object" "SecureString" "String" (required)

AmazonMWSObjectDataset

Name	Description	Value
type	Type of dataset.	"AmazonMWSObject" (required)

AmazonS3Dataset

Name	Description	Value
type	Type of dataset.	"AmazonS3Object" (required)
typeProperties	Amazon S3 dataset properties.	AmazonS3DatasetTypeProperties (required)

AmazonS3DatasetTypeProperties

Name	Description	Value
bucketName	The name of the Amazon S3 bucket. Type: string (or Expression with resultType string).
compression	The data compression method used for the Amazon S3 object.	DatasetCompression
format	The format of files.	DatasetStorageFormat
key	The key of the Amazon S3 object. Type: string (or Expression with resultType string).
prefix	The prefix filter for the S3 object name. Type: string (or Expression with resultType string).
version	The version for the S3 object. Type: string (or Expression with resultType string).

DatasetCompression

Name	Description	Value
type	Set the object type	BZip2 Deflate GZip ZipDeflate (required)

DatasetBZip2Compression

Name	Description	Value
type	Type of dataset compression.	"BZip2" (required)

DatasetDeflateCompression

Name	Description	Value
type	Type of dataset compression.	"Deflate" (required)
level	The Deflate compression level.	"Fastest" "Optimal"

DatasetGZipCompression

Name	Description	Value
type	Type of dataset compression.	"GZip" (required)
level	The GZip compression level.	"Fastest" "Optimal"

DatasetZipDeflateCompression

Name	Description	Value
type	Type of dataset compression.	"ZipDeflate" (required)
level	The ZipDeflate compression level.	"Fastest" "Optimal"

DatasetStorageFormat

Name	Description	Value
deserializer	Deserializer. Type: string (or Expression with resultType string).
serializer	Serializer. Type: string (or Expression with resultType string).

AzureBlobDataset

Name	Description	Value
type	Type of dataset.	"AzureBlob" (required)
typeProperties	Azure Blob dataset properties.	AzureBlobDatasetTypeProperties (required)

AzureBlobDatasetTypeProperties

Name	Description	Value
compression	The data compression method used for the blob storage.	DatasetCompression
fileName	The name of the Azure Blob. Type: string (or Expression with resultType string).
folderPath	The path of the Azure Blob storage. Type: string (or Expression with resultType string).
format	The format of the Azure Blob storage.	DatasetStorageFormat
tableRootLocation	The root of blob path. Type: string (or Expression with resultType string).

AzureDataLakeStoreDataset

Name	Description	Value
type	Type of dataset.	"AzureDataLakeStoreFile" (required)
typeProperties	Azure Data Lake Store dataset properties.	AzureDataLakeStoreDatasetTypeProperties (required)

AzureDataLakeStoreDatasetTypeProperties

Name	Description	Value
compression	The data compression method used for the item(s) in the Azure Data Lake Store.	DatasetCompression
fileName	The name of the file in the Azure Data Lake Store. Type: string (or Expression with resultType string).
folderPath	Path to the folder in the Azure Data Lake Store. Type: string (or Expression with resultType string).
format	The format of the Data Lake Store.	DatasetStorageFormat

AzureMySqlTableDataset

Name	Description	Value
type	Type of dataset.	"AzureMySqlTable" (required)
typeProperties	Azure MySQL database dataset properties.	AzureMySqlTableDatasetTypeProperties (required)

AzureMySqlTableDatasetTypeProperties

Name	Description	Value
tableName	The Azure MySQL database table name. Type: string (or Expression with resultType string).

AzurePostgreSqlTableDataset

Name	Description	Value
type	Type of dataset.	"AzurePostgreSqlTable" (required)

AzureSearchIndexDataset

Name	Description	Value
type	Type of dataset.	"AzureSearchIndex" (required)
typeProperties	Properties specific to this dataset type.	AzureSearchIndexDatasetTypeProperties (required)

AzureSearchIndexDatasetTypeProperties

Name	Description	Value
indexName	The name of the Azure Search Index. Type: string (or Expression with resultType string).

AzureSqlDWTableDataset

Name	Description	Value
type	Type of dataset.	"AzureSqlDWTable" (required)
typeProperties	Azure SQL Data Warehouse dataset properties.	AzureSqlDWTableDatasetTypeProperties (required)

AzureSqlDWTableDatasetTypeProperties

Name	Description	Value
tableName	The table name of the Azure SQL Data Warehouse. Type: string (or Expression with resultType string).

AzureSqlTableDataset

Name	Description	Value
type	Type of dataset.	"AzureSqlTable" (required)
typeProperties	Azure SQL dataset properties.	AzureSqlTableDatasetTypeProperties (required)

AzureSqlTableDatasetTypeProperties

Name	Description	Value
tableName	The table name of the Azure SQL database. Type: string (or Expression with resultType string).

AzureTableDataset

Name	Description	Value
type	Type of dataset.	"AzureTable" (required)
typeProperties	Azure Table dataset properties.	AzureTableDatasetTypeProperties (required)

AzureTableDatasetTypeProperties

Name	Description	Value
tableName	The table name of the Azure Table storage. Type: string (or Expression with resultType string).

CassandraTableDataset

Name	Description	Value
type	Type of dataset.	"CassandraTable" (required)
typeProperties	Cassandra dataset properties.	CassandraTableDatasetTypeProperties (required)

CassandraTableDatasetTypeProperties

Name	Description	Value
keyspace	The keyspace of the Cassandra database. Type: string (or Expression with resultType string).
tableName	The table name of the Cassandra database. Type: string (or Expression with resultType string).

ConcurObjectDataset

Name	Description	Value
type	Type of dataset.	"ConcurObject" (required)

CouchbaseTableDataset

Name	Description	Value
type	Type of dataset.	"CouchbaseTable" (required)

DocumentDbCollectionDataset

Name	Description	Value
type	Type of dataset.	"DocumentDbCollection" (required)
typeProperties	DocumentDB Collection dataset properties.	DocumentDbCollectionDatasetTypeProperties (required)

DocumentDbCollectionDatasetTypeProperties

Name	Description	Value
collectionName	Document Database collection name. Type: string (or Expression with resultType string).

DrillTableDataset

Name	Description	Value
type	Type of dataset.	"DrillTable" (required)

DynamicsEntityDataset

Name	Description	Value
type	Type of dataset.	"DynamicsEntity" (required)
typeProperties	Dynamics entity dataset properties.	DynamicsEntityDatasetTypeProperties (required)

DynamicsEntityDatasetTypeProperties

Name	Description	Value
entityName	The logical name of the entity. Type: string (or Expression with resultType string).

EloquaObjectDataset

Name	Description	Value
type	Type of dataset.	"EloquaObject" (required)

FileShareDataset

Name	Description	Value
type	Type of dataset.	"FileShare" (required)
typeProperties	On-premises file system dataset properties.	FileShareDatasetTypeProperties (required)

FileShareDatasetTypeProperties

Name	Description	Value
compression	The data compression method used for the file system.	DatasetCompression
fileFilter	Specify a filter to be used to select a subset of files in the folderPath rather than all files. Type: string (or Expression with resultType string).
fileName	The name of the on-premises file system. Type: string (or Expression with resultType string).
folderPath	The path of the on-premises file system. Type: string (or Expression with resultType string).
format	The format of the files.	DatasetStorageFormat

GoogleBigQueryObjectDataset

Name	Description	Value
type	Type of dataset.	"GoogleBigQueryObject" (required)

GreenplumTableDataset

Name	Description	Value
type	Type of dataset.	"GreenplumTable" (required)

HBaseObjectDataset

Name	Description	Value
type	Type of dataset.	"HBaseObject" (required)

HiveObjectDataset

Name	Description	Value
type	Type of dataset.	"HiveObject" (required)

HttpDataset

Name	Description	Value
type	Type of dataset.	"HttpFile" (required)
typeProperties	Properties specific to this dataset type.	HttpDatasetTypeProperties (required)

HttpDatasetTypeProperties

Name	Description	Value
additionalHeaders	The headers for the HTTP Request. e.g. request-header-name-1:request-header-value-1 ... request-header-name-n:request-header-value-n Type: string (or Expression with resultType string).
compression	The data compression method used on files.	DatasetCompression
format	The format of files.	DatasetStorageFormat
relativeUrl	The relative URL based on the URL in the HttpLinkedService refers to an HTTP file Type: string (or Expression with resultType string).
requestBody	The body for the HTTP request. Type: string (or Expression with resultType string).
requestMethod	The HTTP method for the HTTP request. Type: string (or Expression with resultType string).

HubspotObjectDataset

Name	Description	Value
type	Type of dataset.	"HubspotObject" (required)

ImpalaObjectDataset

Name	Description	Value
type	Type of dataset.	"ImpalaObject" (required)

JiraObjectDataset

Name	Description	Value
type	Type of dataset.	"JiraObject" (required)

MagentoObjectDataset

Name	Description	Value
type	Type of dataset.	"MagentoObject" (required)

MariaDBTableDataset

Name	Description	Value
type	Type of dataset.	"MariaDBTable" (required)

MarketoObjectDataset

Name	Description	Value
type	Type of dataset.	"MarketoObject" (required)

MongoDbCollectionDataset

Name	Description	Value
type	Type of dataset.	"MongoDbCollection" (required)
typeProperties	MongoDB database dataset properties.	MongoDbCollectionDatasetTypeProperties (required)

MongoDbCollectionDatasetTypeProperties

Name	Description	Value
collectionName	The table name of the MongoDB database. Type: string (or Expression with resultType string).

NetezzaTableDataset

Name	Description	Value
type	Type of dataset.	"NetezzaTable" (required)

ODataResourceDataset

Name	Description	Value
type	Type of dataset.	"ODataResource" (required)
typeProperties	OData dataset properties.	ODataResourceDatasetTypeProperties (required)

ODataResourceDatasetTypeProperties

Name	Description	Value
path	The OData resource path. Type: string (or Expression with resultType string).

OracleTableDataset

Name	Description	Value
type	Type of dataset.	"OracleTable" (required)
typeProperties	On-premises Oracle dataset properties.	OracleTableDatasetTypeProperties (required)

OracleTableDatasetTypeProperties

Name	Description	Value
tableName	The table name of the on-premises Oracle database. Type: string (or Expression with resultType string).

PaypalObjectDataset

Name	Description	Value
type	Type of dataset.	"PaypalObject" (required)

PhoenixObjectDataset

Name	Description	Value
type	Type of dataset.	"PhoenixObject" (required)

PrestoObjectDataset

Name	Description	Value
type	Type of dataset.	"PrestoObject" (required)

QuickBooksObjectDataset

Name	Description	Value
type	Type of dataset.	"QuickBooksObject" (required)

RelationalTableDataset

Name	Description	Value
type	Type of dataset.	"RelationalTable" (required)
typeProperties	Relational table dataset properties.	RelationalTableDatasetTypeProperties (required)

RelationalTableDatasetTypeProperties

Name	Description	Value
tableName	The relational table name. Type: string (or Expression with resultType string).

ResponsysObjectDataset

Name	Description	Value
type	Type of dataset.	"ResponsysObject" (required)

SalesforceMarketingCloudObjectDataset

Name	Description	Value
type	Type of dataset.	"SalesforceMarketingCloudObject" (required)

SalesforceObjectDataset

Name	Description	Value
type	Type of dataset.	"SalesforceObject" (required)
typeProperties	Salesforce object dataset properties.	SalesforceObjectDatasetTypeProperties (required)

SalesforceObjectDatasetTypeProperties

Name	Description	Value
objectApiName	The Salesforce object API name. Type: string (or Expression with resultType string).

SapCloudForCustomerResourceDataset

Name	Description	Value
type	Type of dataset.	"SapCloudForCustomerResource" (required)
typeProperties	SAP Cloud For Customer OData resource dataset properties.	SapCloudForCustomerResourceDatasetTypeProperties (required)

SapCloudForCustomerResourceDatasetTypeProperties

Name	Description	Value
path	The path of the SAP Cloud for Customer OData entity. Type: string (or Expression with resultType string).

SapEccResourceDataset

Name	Description	Value
type	Type of dataset.	"SapEccResource" (required)
typeProperties	SAP ECC OData resource dataset properties.	SapEccResourceDatasetTypeProperties (required)

SapEccResourceDatasetTypeProperties

Name	Description	Value
path	The path of the SAP ECC OData entity. Type: string (or Expression with resultType string).

ServiceNowObjectDataset

Name	Description	Value
type	Type of dataset.	"ServiceNowObject" (required)

ShopifyObjectDataset

Name	Description	Value
type	Type of dataset.	"ShopifyObject" (required)

SparkObjectDataset

Name	Description	Value
type	Type of dataset.	"SparkObject" (required)

SqlServerTableDataset

Name	Description	Value
type	Type of dataset.	"SqlServerTable" (required)
typeProperties	On-premises SQL Server dataset properties.	SqlServerTableDatasetTypeProperties (required)

SqlServerTableDatasetTypeProperties

Name	Description	Value
tableName	The table name of the SQL Server dataset. Type: string (or Expression with resultType string).

SquareObjectDataset

Name	Description	Value
type	Type of dataset.	"SquareObject" (required)

VerticaTableDataset

Name	Description	Value
type	Type of dataset.	"VerticaTable" (required)

WebTableDataset

Name	Description	Value
type	Type of dataset.	"WebTable" (required)
typeProperties	Web table dataset properties.	WebTableDatasetTypeProperties (required)

WebTableDatasetTypeProperties

Name	Description	Value
index	The zero-based index of the table in the web page. Type: integer (or Expression with resultType integer), minimum: 0.
path	The relative URL to the web page from the linked service URL. Type: string (or Expression with resultType string).

XeroObjectDataset

Name	Description	Value
type	Type of dataset.	"XeroObject" (required)

ZohoObjectDataset

Name	Description	Value
type	Type of dataset.	"ZohoObject" (required)

Share via

Microsoft.DataFactory factories/datasets 2017-09-01-preview

Bicep resource definition

Resource format

Dataset objects

DatasetCompression objects

DatasetStorageFormat objects

Property values

factories/datasets

Dataset

LinkedServiceReference

ParameterValueSpecification

ParameterDefinitionSpecification

ParameterSpecification

AmazonMWSObjectDataset

AmazonS3Dataset

AmazonS3DatasetTypeProperties

DatasetCompression

DatasetBZip2Compression

DatasetDeflateCompression

DatasetGZipCompression

DatasetZipDeflateCompression

DatasetStorageFormat

AzureBlobDataset

AzureBlobDatasetTypeProperties

AzureDataLakeStoreDataset

AzureDataLakeStoreDatasetTypeProperties

AzureMySqlTableDataset

AzureMySqlTableDatasetTypeProperties

AzurePostgreSqlTableDataset

AzureSearchIndexDataset

AzureSearchIndexDatasetTypeProperties

AzureSqlDWTableDataset

AzureSqlDWTableDatasetTypeProperties

AzureSqlTableDataset

AzureSqlTableDatasetTypeProperties

AzureTableDataset

AzureTableDatasetTypeProperties

CassandraTableDataset

CassandraTableDatasetTypeProperties

ConcurObjectDataset

CouchbaseTableDataset

DocumentDbCollectionDataset

DocumentDbCollectionDatasetTypeProperties

DrillTableDataset

DynamicsEntityDataset

DynamicsEntityDatasetTypeProperties

EloquaObjectDataset

FileShareDataset

FileShareDatasetTypeProperties

GoogleBigQueryObjectDataset

GreenplumTableDataset

HBaseObjectDataset

HiveObjectDataset

HttpDataset

HttpDatasetTypeProperties

HubspotObjectDataset

ImpalaObjectDataset

JiraObjectDataset

MagentoObjectDataset

MariaDBTableDataset

MarketoObjectDataset

MongoDbCollectionDataset

MongoDbCollectionDatasetTypeProperties

NetezzaTableDataset

ODataResourceDataset

ODataResourceDatasetTypeProperties

OracleTableDataset

OracleTableDatasetTypeProperties

PaypalObjectDataset

PhoenixObjectDataset

PrestoObjectDataset

QuickBooksObjectDataset

RelationalTableDataset

RelationalTableDatasetTypeProperties

ResponsysObjectDataset

SalesforceMarketingCloudObjectDataset

SalesforceObjectDataset

SalesforceObjectDatasetTypeProperties

SapCloudForCustomerResourceDataset