Big Data Pools - Create Or Update

Reference

Service:: Synapse

API Version:: 2021-06-01

Create a Big Data pool.
Create a new Big Data pool.

PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Synapse/workspaces/{workspaceName}/bigDataPools/{bigDataPoolName}?api-version=2021-06-01

With optional parameters:

PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Synapse/workspaces/{workspaceName}/bigDataPools/{bigDataPoolName}?api-version=2021-06-01&force={force}

URI Parameters

Name	In	Required	Type	Description
bigDataPoolName	path	True	string	Big Data pool name
resourceGroupName	path	True	string	The name of the resource group. The name is case insensitive.
subscriptionId	path	True	string	The ID of the target subscription.
workspaceName	path	True	string	The name of the workspace.
api-version	query	True	string	The API version to use for this operation.
force	query		boolean	Whether to stop any running jobs in the Big Data pool

Request Body

Name	Required	Type	Description
location	True	string	The geo-location where the resource lives
properties.autoPause		AutoPauseProperties	Spark pool auto-pausing properties Auto-pausing properties
properties.autoScale		AutoScaleProperties	Spark pool auto-scaling properties Auto-scaling properties
properties.cacheSize		integer	The cache size
properties.customLibraries		LibraryInfo[]	List of custom libraries/packages associated with the spark pool.
properties.defaultSparkLogFolder		string	The default folder where Spark logs will be written.
properties.dynamicExecutorAllocation		DynamicExecutorAllocation	Dynamic Executor Allocation
properties.isAutotuneEnabled		boolean	Enable Autotune Whether autotune is required or not.
properties.isComputeIsolationEnabled		boolean	Whether compute isolation is required or not.
properties.libraryRequirements		LibraryRequirements	Spark pool library version requirements Library version requirements
properties.nodeCount		integer	The number of nodes in the Big Data pool.
properties.nodeSize		NodeSize	The level of compute power that each node in the Big Data pool has.
properties.nodeSizeFamily		NodeSizeFamily	The kind of nodes that the Big Data pool provides.
properties.provisioningState		string	The state of the Big Data pool.
properties.sessionLevelPackagesEnabled		boolean	Whether session level packages enabled.
properties.sparkConfigProperties		SparkConfigProperties	Spark pool Config Properties Spark configuration file to specify additional properties
properties.sparkEventsFolder		string	The Spark events folder
properties.sparkVersion		string	The Apache Spark version.
tags		object	Resource tags.

Responses

Name	Type	Description
200 OK	BigDataPoolResourceInfo	OK
202 Accepted	BigDataPoolResourceInfo
Other Status Codes	ErrorResponse

Examples

Create or update a Big Data pool

Sample request

HTTP

PUT https://management.azure.com/subscriptions/01234567-89ab-4def-0123-456789abcdef/resourceGroups/ExampleResourceGroup/providers/Microsoft.Synapse/workspaces/ExampleWorkspace/bigDataPools/ExamplePool?api-version=2021-06-01

{
  "tags": {
    "key": "value"
  },
  "location": "West US 2",
  "properties": {
    "sparkVersion": "3.3",
    "nodeCount": 4,
    "nodeSize": "Medium",
    "nodeSizeFamily": "MemoryOptimized",
    "autoScale": {
      "enabled": true,
      "minNodeCount": 3,
      "maxNodeCount": 50
    },
    "autoPause": {
      "enabled": true,
      "delayInMinutes": 15
    },
    "sparkEventsFolder": "/events",
    "libraryRequirements": {
      "content": "",
      "filename": "requirements.txt"
    },
    "defaultSparkLogFolder": "/logs",
    "isAutotuneEnabled": false
  }
}

Sample response

Status code:: 200

{
  "id": "/subscriptions/01234567-89ab-4def-0123-456789abcdef/resourceGroups/ExampleResourceGroup/providers/Microsoft.Synapse/workspaces/ExampleWorkspace/bigDataPools/ExamplePool",
  "type": "Microsoft.Synapse/workspaces/bigDataPools",
  "location": "West US 2",
  "name": "ExamplePool",
  "tags": {
    "key": "value"
  },
  "properties": {
    "provisioningState": "Provisioning",
    "sparkVersion": "3.3",
    "nodeCount": 4,
    "nodeSize": "Medium",
    "nodeSizeFamily": "MemoryOptimized",
    "autoScale": {
      "enabled": true,
      "minNodeCount": 3,
      "maxNodeCount": 50
    },
    "autoPause": {
      "enabled": true,
      "delayInMinutes": 15
    },
    "creationDate": "1970-01-01T00:00:00Z",
    "sparkEventsFolder": "/events",
    "libraryRequirements": {
      "time": "1970-01-01T00:00:00Z",
      "content": "",
      "filename": "requirements.txt"
    },
    "defaultSparkLogFolder": "/logs",
    "lastSucceededTimestamp": "1970-01-01T10:00:00Z",
    "isAutotuneEnabled": false
  }
}

Status code:: 202

{
  "id": "/subscriptions/01234567-89ab-4def-0123-456789abcdef/resourceGroups/ExampleResourceGroup/providers/Microsoft.Synapse/workspaces/ExampleWorkspace/bigDataPools/ExamplePool",
  "type": "Microsoft.Synapse/workspaces/bigDataPools",
  "location": "West US 2",
  "name": "ExamplePool",
  "tags": {
    "key": "value"
  },
  "properties": {
    "provisioningState": "Provisioning",
    "sparkVersion": "3.3",
    "nodeCount": 4,
    "nodeSize": "Medium",
    "nodeSizeFamily": "MemoryOptimized",
    "autoScale": {
      "enabled": true,
      "minNodeCount": 3,
      "maxNodeCount": 50
    },
    "autoPause": {
      "enabled": true,
      "delayInMinutes": 15
    },
    "creationDate": "1970-01-01T00:00:00Z",
    "sparkEventsFolder": "/events",
    "libraryRequirements": {
      "time": "1970-01-01T00:00:00Z",
      "content": "",
      "filename": "requirements.txt"
    },
    "defaultSparkLogFolder": "/logs",
    "isAutotuneEnabled": false
  }
}

Status code:: default

{
  "error": {
    "code": "Error code",
    "message": "Error message"
  }
}

Definitions

Name	Description
AutoPauseProperties	Spark pool auto-pausing properties
AutoScaleProperties	Spark pool auto-scaling properties
BigDataPoolResourceInfo	Big Data pool
ConfigurationType	The type of the spark config properties file.
DynamicExecutorAllocation	Dynamic Executor Allocation Properties
ErrorAdditionalInfo	The resource management error additional info.
ErrorDetail	The error detail.
ErrorResponse	Error response
LibraryInfo	Information about a library/package created at the workspace level.
LibraryRequirements	Spark pool library version requirements
NodeSize	The level of compute power that each node in the Big Data pool has.
NodeSizeFamily	The kind of nodes that the Big Data pool provides.
SparkConfigProperties	Spark pool Config Properties

AutoPauseProperties

Spark pool auto-pausing properties

Name	Type	Description
delayInMinutes	integer	Number of minutes of idle time before the Big Data pool is automatically paused.
enabled	boolean	Whether auto-pausing is enabled for the Big Data pool.

AutoScaleProperties

Spark pool auto-scaling properties

Name	Type	Description
enabled	boolean	Whether automatic scaling is enabled for the Big Data pool.
maxNodeCount	integer	The maximum number of nodes the Big Data pool can support.
minNodeCount	integer	The minimum number of nodes the Big Data pool can support.

BigDataPoolResourceInfo

Big Data pool

Name	Type	Description
id	string	Fully qualified resource ID for the resource. Ex - /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/{resourceProviderNamespace}/{resourceType}/{resourceName}
location	string	The geo-location where the resource lives
name	string	The name of the resource
properties.autoPause	AutoPauseProperties	Spark pool auto-pausing properties Auto-pausing properties
properties.autoScale	AutoScaleProperties	Spark pool auto-scaling properties Auto-scaling properties
properties.cacheSize	integer	The cache size
properties.creationDate	string	The time when the Big Data pool was created.
properties.customLibraries	LibraryInfo[]	List of custom libraries/packages associated with the spark pool.
properties.defaultSparkLogFolder	string	The default folder where Spark logs will be written.
properties.dynamicExecutorAllocation	DynamicExecutorAllocation	Dynamic Executor Allocation
properties.isAutotuneEnabled	boolean	Enable Autotune Whether autotune is required or not.
properties.isComputeIsolationEnabled	boolean	Whether compute isolation is required or not.
properties.lastSucceededTimestamp	string	The time when the Big Data pool was updated successfully.
properties.libraryRequirements	LibraryRequirements	Spark pool library version requirements Library version requirements
properties.nodeCount	integer	The number of nodes in the Big Data pool.
properties.nodeSize	NodeSize	The level of compute power that each node in the Big Data pool has.
properties.nodeSizeFamily	NodeSizeFamily	The kind of nodes that the Big Data pool provides.
properties.provisioningState	string	The state of the Big Data pool.
properties.sessionLevelPackagesEnabled	boolean	Whether session level packages enabled.
properties.sparkConfigProperties	SparkConfigProperties	Spark pool Config Properties Spark configuration file to specify additional properties
properties.sparkEventsFolder	string	The Spark events folder
properties.sparkVersion	string	The Apache Spark version.
tags	object	Resource tags.
type	string	The type of the resource. E.g. "Microsoft.Compute/virtualMachines" or "Microsoft.Storage/storageAccounts"

ConfigurationType

The type of the spark config properties file.

Name	Type	Description
Artifact	string
File	string

DynamicExecutorAllocation

Dynamic Executor Allocation Properties

Name	Type	Description
enabled	boolean	Indicates whether Dynamic Executor Allocation is enabled or not.
maxExecutors	integer	The maximum number of executors alloted
minExecutors	integer	The minimum number of executors alloted

ErrorAdditionalInfo

The resource management error additional info.

Name	Type	Description
info	object	The additional info.
type	string	The additional info type.

ErrorDetail

The error detail.

Name	Type	Description
additionalInfo	ErrorAdditionalInfo[]	The error additional info.
code	string	The error code.
details	ErrorDetail[]	The error details.
message	string	The error message.
target	string	The error target.

ErrorResponse

Error response

Name	Type	Description
error	ErrorDetail	The error object.

LibraryInfo

Information about a library/package created at the workspace level.

Name	Type	Description
containerName	string	Storage blob container name.
creatorId	string	Creator Id of the library/package.
name	string	Name of the library.
path	string	Storage blob path of library.
provisioningStatus	string	Provisioning status of the library/package.
type	string	Type of the library.
uploadedTimestamp	string	The last update time of the library.

LibraryRequirements

Spark pool library version requirements

Name	Type	Description
content	string	The library requirements.
filename	string	The filename of the library requirements file.
time	string	The last update time of the library requirements file.

NodeSize

The level of compute power that each node in the Big Data pool has.

Name	Type	Description
Large	string
Medium	string
None	string
Small	string
XLarge	string
XXLarge	string
XXXLarge	string

NodeSizeFamily

The kind of nodes that the Big Data pool provides.

Name	Type	Description
HardwareAcceleratedFPGA	string
HardwareAcceleratedGPU	string
MemoryOptimized	string
None	string

SparkConfigProperties

Spark pool Config Properties

Name	Type	Description
configurationType	ConfigurationType	The type of the spark config properties file.
content	string	The spark config properties.
filename	string	The filename of the spark config properties file.
time	string	The last update time of the spark config properties file.

Compartilhar via