This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 65%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
Has anyone experienced running Spark on Azure Kubernetes Service (AKS)? I'm encountering an issue where the Spark driver starts successfully, but it fails to launch executor pods for the Spark job. Any insights would be appreciated. I am running a azure managed control plane (default i suppose).
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(32, 3%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMG%3C/text%3E%3C/svg%3E)
Hi Bhaskar Marthi,
Thank you for reaching out to the Microsoft Q&A platform.
Based on your query, could you please confirm if you are referring to Apache Spark or another application.
If you are referring Apache Spark, please find the documentation here
If you are facing any issue, please share me the error screenshots, so that I can assist you better.
Thanks Mahesh for the pointer. I did use that documentation as a guideline though that was written 3 years ago using spark 2.3 (current spark is at 3.5). My cluster also has azure RBAC enabled for which i followed necessary steps to create role and rolebinding. The exception i see is the following:
ERROR spark.SparkContext: Error initializing SparkContext.
org.apache.spark.SparkException: External scheduler cannot be instantiated
at org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:2979)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:559)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2690)
at org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$2(SparkSession.scala:949)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:943)
at org.apache.spark.examples.SparkPi$.main(SparkPi.scala:30)
at org.apache.spark.examples.SparkPi.main(SparkPi.scala)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Operation: [get] for kind: [Pod] with name: [spark-pi] in namespace: [MYNAMESPACE] failed.
This indicates potentially two problems.
So trying to figure out what is the issue with API server access in AKS.
Hello Bhaskar Marthi,
Just checking in to see if you have got a chance to see the comment posted in resolving the issue.
If it was helpful, please click "Upvote" on this post to let us know.
Thank You.
Hello Bhaskar Marthi,
Thank you for your response.
Based on error message provided, please check that your Kubernetes cluster is correctly configured and accessible from your Spark application with necessary permissions are set for the Spark driver to communicate.
Check that your Kubernetes cluster is properly configured, and version is compatible with your Spark version, for Spark 3.5.3, it is recommended to use Kubernetes version 1.24 or higher.
https://spark.apache.org/docs/3.5.3/running-on-kubernetes.html
Check your cluster has sufficient resources (CPU and memory) allocated to run Spark applications effectively.
The Spark application is configured to use the correct namespace. The namespace should match where your Spark resources are deployed.
Please find the below similar issue article for your reference:
https://github.com/kubeflow/spark-operator/issues/1562
I hope this information is helpful. Please feel free to reach out if you have any further questions.
If the answer is helpful, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".
Hello Bhaskar Marthi,
I wanted to check if you had the opportunity to review the information which was provided in my previous posted answer.
If you feel that your quires have been resolved, please accept the answer by clicking the "Upvote" and "Accept Answer" on the post.
Thank you.