Items - Create Spark Job Definition

Reference

Service:: SparkJobDefinition

API Version:: v1

Creates a spark job definition in the specified workspace.
This API supports long running operations (LRO).

To create spark job definition with a public definition, refer to Spark job definition article.

Permissions

The caller must have contributor or higher workspace role.

Required Delegated Scopes

SparkJobDefinition.ReadWrite.All or Item.ReadWrite.All

Limitations

To create a spark job definition the workspace must be on a supported Fabric capacity. For more information see: Microsoft Fabric license types.

Microsoft Entra supported identities

This API supports the Microsoft identities listed in this section.

Identity	Support
User	Yes
Service principal	Yes
Managed identities	Yes

Interface

POST https://api.fabric.microsoft.com/v1/workspaces/{workspaceId}/sparkJobDefinitions

URI Parameters

Name	In	Required	Type	Description
workspaceId	path	True	string uuid	The workspace ID.

Request Body

Name	Required	Type	Description
displayName	True	string	The spark job definition display name. The display name must follow naming rules according to item type.
definition		SparkJobDefinitionPublicDefinition	The spark job definition public definition.
description		string	The spark job definition description. Maximum length is 256 characters.

Responses

Name	Type	Description
201 Created	SparkJobDefinition	Successfully created
202 Accepted		Request accepted, spark job definition provisioning in progress. Headers Location: string x-ms-operation-id: string Retry-After: integer
Other Status Codes	ErrorResponse	Common error codes: InvalidItemType - Item type is invalid ItemDisplayNameAlreadyInUse - Item display name is already used. CorruptedPayload - The provided payload is corrupted.

Name

Type

Description

201 Created

SparkJobDefinition

Successfully created

202 Accepted

Request accepted, spark job definition provisioning in progress.

Headers

Location: string
x-ms-operation-id: string
Retry-After: integer

Other Status Codes

ErrorResponse

Common error codes:

InvalidItemType - Item type is invalid
ItemDisplayNameAlreadyInUse - Item display name is already used.
CorruptedPayload - The provided payload is corrupted.

Examples

Create a spark job definition example

Create a spark job definition with public definition example

Create a spark job definition example

Sample request

HTTP

POST https://api.fabric.microsoft.com/v1/workspaces/cfafbeb1-8037-4d0c-896e-a46fb27ff229/sparkJobDefinitions

{
  "displayName": "SparkJobDefinition 1",
  "description": "A spark job definition description."
}

Sample response

Status code:: 201

{
  "displayName": "SparkJobDefinition 1",
  "description": "A spark job definition description.",
  "type": "SparkJobDefinition",
  "workspaceId": "cfafbeb1-8037-4d0c-896e-a46fb27ff229",
  "id": "5b218778-e7a5-4d73-8187-f10824047715"
}

Status code:: 202

Location: https://api.fabric.microsoft.com/v1/operations/0acd697c-1550-43cd-b998-91bfbfbd47c6
x-ms-operation-id: 0acd697c-1550-43cd-b998-91bfbfbd47c6
Retry-After: 30

Create a spark job definition with public definition example

Sample request

HTTP

POST https://api.fabric.microsoft.com/v1/workspaces/cfafbeb1-8037-4d0c-896e-a46fb27ff229/sparkJobDefinitions

{
  "displayName": "SparkJobDefinition 1",
  "description": "A spark job definition description.",
  "definition": {
    "format": "SparkJobDefinitionV1",
    "parts": [
      {
        "path": "SparkJobDefinitionV1.json",
        "payload": "eyJleGVjdXRhYmxlRm..OWRmNDhhY2ZmZTgifQ==",
        "payloadType": "InlineBase64"
      },
      {
        "path": ".platform",
        "payload": "ZG90UGxhdGZvcm1CYXNlNjRTdHJpbmc=",
        "payloadType": "InlineBase64"
      }
    ]
  }
}

Sample response

Status code:: 201

{
  "displayName": "SparkJobDefinition 1",
  "description": "A spark job definition description.",
  "type": "SparkJobDefinition",
  "workspaceId": "cfafbeb1-8037-4d0c-896e-a46fb27ff229",
  "id": "5b218778-e7a5-4d73-8187-f10824047715"
}

Status code:: 202

Location: https://api.fabric.microsoft.com/v1/operations/0acd697c-1550-43cd-b998-91bfbfbd47c6
x-ms-operation-id: 0acd697c-1550-43cd-b998-91bfbfbd47c6
Retry-After: 30

Definitions

Name	Description
CreateSparkJobDefinitionRequest	Create spark job definition request payload.
ErrorRelatedResource	The error related resource details object.
ErrorResponse	The error response.
ErrorResponseDetails	The error response details.
ItemType	The type of the item. Additional item types may be added over time.
PayloadType	The type of the definition part payload. Additional payload types may be added over time.
SparkJobDefinition	A spark job definition object.
SparkJobDefinitionProperties	The spark job definition properties.
SparkJobDefinitionPublicDefinition	Spark job definition public definition object. Refer to this article for more details on how to craft a spark job definition public definition.
SparkJobDefinitionPublicDefinitionPart	Spark job definition definition part object.

CreateSparkJobDefinitionRequest

Create spark job definition request payload.

Name	Type	Description
definition	SparkJobDefinitionPublicDefinition	The spark job definition public definition.
description	string	The spark job definition description. Maximum length is 256 characters.
displayName	string	The spark job definition display name. The display name must follow naming rules according to item type.

ErrorRelatedResource

The error related resource details object.

Name	Type	Description
resourceId	string	The resource ID that's involved in the error.
resourceType	string	The type of the resource that's involved in the error.

ErrorResponse

The error response.

Name	Type	Description
errorCode	string	A specific identifier that provides information about an error condition, allowing for standardized communication between our service and its users.
message	string	A human readable representation of the error.
moreDetails	ErrorResponseDetails[]	List of additional error details.
relatedResource	ErrorRelatedResource	The error related resource details.
requestId	string	ID of the request associated with the error.

ErrorResponseDetails

The error response details.

Name	Type	Description
errorCode	string	A specific identifier that provides information about an error condition, allowing for standardized communication between our service and its users.
message	string	A human readable representation of the error.
relatedResource	ErrorRelatedResource	The error related resource details.

ItemType

The type of the item. Additional item types may be added over time.

Name	Type	Description
Dashboard	string	PowerBI dashboard.
DataPipeline	string	A data pipeline.
Datamart	string	PowerBI datamart.
Environment	string	An environment.
Eventhouse	string	An eventhouse.
Eventstream	string	An eventstream.
KQLDashboard	string	A KQL dashboard.
KQLDatabase	string	A KQL database.
KQLQueryset	string	A KQL queryset.
Lakehouse	string	A lakehouse.
MLExperiment	string	A machine learning experiment.
MLModel	string	A machine learning model.
MirroredDatabase	string	A mirrored database.
MirroredWarehouse	string	A mirrored warehouse.
Notebook	string	A notebook.
PaginatedReport	string	PowerBI paginated report.
Reflex	string	A Reflex.
Report	string	PowerBI report.
SQLEndpoint	string	An SQL endpoint.
SemanticModel	string	PowerBI semantic model.
SparkJobDefinition	string	A spark job definition.
Warehouse	string	A warehouse.

PayloadType

The type of the definition part payload. Additional payload types may be added over time.

Name	Type	Description
InlineBase64	string	Inline Base 64.

SparkJobDefinition

A spark job definition object.

Name	Type	Description
description	string	The item description.
displayName	string	The item display name.
id	string	The item ID.
properties	SparkJobDefinitionProperties	The spark job definition properties.
type	ItemType	The item type.
workspaceId	string	The workspace ID.

SparkJobDefinitionProperties

The spark job definition properties.

Name	Type	Description
oneLakeRootPath	string	OneLake path to the SparkJobDefinition root directory.

SparkJobDefinitionPublicDefinition

Spark job definition public definition object. Refer to this article for more details on how to craft a spark job definition public definition.

Name	Type	Description
format	string	The format of the item definition. Supported format: `SparkJobDefinitionV1`.
parts	SparkJobDefinitionPublicDefinitionPart[]	A list of definition parts.

SparkJobDefinitionPublicDefinitionPart

Spark job definition definition part object.

Name	Type	Description
path	string	The spark job definition public definition part path.
payload	string	The spark job definition public definition part payload.
payloadType	PayloadType	The payload type.

Jaa

Items - Create Spark Job Definition

Permissions

Required Delegated Scopes

Limitations

Microsoft Entra supported identities

Interface

URI Parameters

Request Body

Responses

Examples

Create a spark job definition example

Sample request

Sample response

Create a spark job definition with public definition example

Sample request

Sample response

Definitions

CreateSparkJobDefinitionRequest

ErrorRelatedResource

ErrorResponse

ErrorResponseDetails

ItemType

PayloadType

SparkJobDefinition

SparkJobDefinitionProperties

SparkJobDefinitionPublicDefinition

SparkJobDefinitionPublicDefinitionPart

Lisäresursseja