JAR task for jobs

Use the JAR task to deploy Scala or Java code compiled into a JAR (Java ARchive).

Important

Scala has support limitations in Unity Catalog standard access mode. See Language support for Unity Catalog standard access mode and Spark API limitations and requirements for Unity Catalog standard access mode.

Standard access mode requires an admin to add Maven coordinates and paths for JAR libraries to an allowlist. See Allowlist libraries and init scripts on compute with standard access mode (formerly shared access mode).

You must store JAR files in a location supported by your compute configurations. See Java and Scala library support.

Requirements

• You must choose a compute configuration that supports Scala workloads.
• You must upload your JAR file to a location or Maven repository compatible with your compute configuration.
• The code in your JAR file must use SparkContext.getOrCreate to obtain a Spark context.

Configure a JAR task

Add a JAR task from the Tasks tab in the Jobs UI by doing the following:

1. In the Type drop-down menu, select JAR.
2. Specify the Main class.
   • This is the full name of the class containing the main method to be executed. This class must be included in a JAR configured as a Dependent library.
3. Use Compute to select or configure a cluster that supports the logic in your JAR.
4. Click + Add under Dependent libraries. The Add dependent library dialogue appears.
   • You can select an existing JAR file or upload a new JAR file.
   • Not all locations support JAR files.
   • Not all compute configurations support JAR files in all supported locations.
   • Each Library Source has a different flow for selecting or uploading a JAR file. See Libraries.
5. (Optional) Configure Parameters as a list of strings passed as arguments to the main class. See Configure task parameters.
6. Click Save task.