Spark Session - Create Spark Session

Tjänst:

Synapse

API-version:

2020-12-01

Skapa en ny Spark-session.

POST {endpoint}/livyApi/versions/{livyApiVersion}/sparkPools/{sparkPoolName}/sessions

Med valfria parametrar:

POST {endpoint}/livyApi/versions/{livyApiVersion}/sparkPools/{sparkPoolName}/sessions?detailed={detailed}

URI-parametrar

Name	I	Obligatorisk	Typ	Description
endpoint	path	True	string	Slutpunkten för arbetsytans utveckling, till exempel https://myworkspace.dev.azuresynapse.net.
livyApiVersion	path	True	string	Giltig API-version för begäran.
sparkPoolName	path	True	string	Namnet på spark-poolen.
detailed	query		boolean	Valfri frågeparam som anger om ett detaljerat svar returneras bortom oformaterad livy.

Begärandetext

Name	Obligatorisk	Typ	Description
name	True	string
archives		string[]
args		string[]
artifactId		string
className		string
conf		object
driverCores		integer (int32)
driverMemory		string
executorCores		integer (int32)
executorMemory		string
file		string
files		string[]
jars		string[]
numExecutors		integer (int32)
pyFiles		string[]
tags		object

Svar

Name	Typ	Description
200 OK	SparkSession	Framgång

Exempel

Create new spark session.

Exempelbegäran

HTTP

POST myWorkspace.dev.azuresynapse.net/livyApi/versions/2019-11-01-preview/sparkPools/mySparkPool/sessions?detailed=True

{
  "tags": null,
  "artifactId": "fill in here",
  "name": "jobname",
  "file": "abfss://",
  "className": "classname",
  "args": [],
  "jars": [],
  "files": [],
  "archives": [],
  "conf": null,
  "driverMemory": "4g",
  "driverCores": 4,
  "executorMemory": "2g",
  "executorCores": 2,
  "numExecutors": 2
}

Exempelsvar

Statuskod:

200

{
  "livyInfo": null,
  "name": "jobname",
  "workspaceName": "myWorkspace",
  "sparkPoolName": "mySparkPool",
  "submitterName": "thetime",
  "submitterId": "thesubmitterid",
  "artifactId": "fill in here",
  "jobType": "SparkSession",
  "result": "Failed",
  "schedulerInfo": null,
  "pluginInfo": null,
  "errorInfo": [],
  "tags": null,
  "id": 1,
  "appId": "fill in here",
  "appInfo": null,
  "state": "the state",
  "log": []
}

Definitioner

Name	Description
LivyStates	Batchtillståndet
PluginCurrentState
SchedulerCurrentState
SparkErrorSource
SparkJobType	Jobbtypen.
SparkRequest
SparkScheduler
SparkServiceError
SparkServicePlugin
SparkSession
SparkSessionOptions
SparkSessionResultType
SparkSessionState

LivyStates

Uppräkning

Batchtillståndet

Värde	Description
busy
dead
error
idle
killed
not_started
recovering
running
shutting_down
starting
success

PluginCurrentState

Uppräkning

Värde	Description
Cleanup
Ended
Monitoring
Preparation
Queued
ResourceAcquisition
Submission

SchedulerCurrentState

Uppräkning

Värde	Description
Ended
Queued
Scheduled

SparkErrorSource

Uppräkning

Värde	Description
Dependency
System
Unknown
User

SparkJobType

Uppräkning

Jobbtypen.

Värde	Description
SparkBatch
SparkSession

SparkRequest

Objekt

Name	Typ	Description
archives	string[]
args	string[]
className	string
conf	object
driverCores	integer (int32)
driverMemory	string
executorCores	integer (int32)
executorMemory	string
file	string
files	string[]
jars	string[]
name	string
numExecutors	integer (int32)
pyFiles	string[]

SparkScheduler

Objekt

Name	Typ	Description
cancellationRequestedAt	string (date-time)
currentState	SchedulerCurrentState
endedAt	string (date-time)
scheduledAt	string (date-time)
submittedAt	string (date-time)

SparkServiceError

Objekt

Name	Typ	Description
errorCode	string
message	string
source	SparkErrorSource

SparkServicePlugin

Objekt

Name	Typ	Description
cleanupStartedAt	string (date-time)
currentState	PluginCurrentState
monitoringStartedAt	string (date-time)
preparationStartedAt	string (date-time)
resourceAcquisitionStartedAt	string (date-time)
submissionStartedAt	string (date-time)

SparkSession

Objekt

Name	Typ	Description
appId	string
appInfo	object
artifactId	string
errorInfo	SparkServiceError[]
id	integer (int32)
jobType	SparkJobType
livyInfo	SparkSessionState
log	string[]
name	string
pluginInfo	SparkServicePlugin
result	SparkSessionResultType
schedulerInfo	SparkScheduler
sparkPoolName	string
state	LivyStates	Sessionstillståndet.
submitterId	string
submitterName	string
tags	object
workspaceName	string

SparkSessionOptions

Objekt

Name	Typ	Description
archives	string[]
args	string[]
artifactId	string
className	string
conf	object
driverCores	integer (int32)
driverMemory	string
executorCores	integer (int32)
executorMemory	string
file	string
files	string[]
jars	string[]
name	string
numExecutors	integer (int32)
pyFiles	string[]
tags	object

SparkSessionResultType

Uppräkning

Värde	Description
Cancelled
Failed
Succeeded
Uncertain

SparkSessionState

Objekt

Name	Typ	Description
busyAt	string (date-time)
currentState	string
deadAt	string (date-time)
errorAt	string (date-time)
idleAt	string (date-time)
jobCreationRequest	SparkRequest
killedAt	string (date-time)
notStartedAt	string (date-time)
recoveringAt	string (date-time)
shuttingDownAt	string (date-time)
startingAt	string (date-time)