Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,283 questions
2 answers
One of the answers was accepted by the question author.
Azure Databricks Usage Cost
Hi Team, We have an Azure Databricks subscription with reservations, and multiple workspaces for different projects. We want to monitor our cost usage in the best way possible. I've explored two options: Creating a consumption dashboard in the Account…
asked 2024-12-18T16:43:33.6566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(38.4, 15%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMT%3C/text%3E%3C/svg%3E)
Mukesh Thakor
40
Reputation points
commented 2024-12-22T02:14:45.2833333+00:00
Mukesh Thakor
40
Reputation points
1 answer
How can you see the user that is assigned the Role in an ARM (Bicep) deployment.
I have this Bicep code which creates an Azure Databricks access connector and then gives Reader Role to some Service Principal that have been given as an input in the parameters. resource accessConnector…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(121.6, 83%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
1 answer
One of the answers was accepted by the question author.
Unable to Enable Serverless Clusters in Azure Databricks Workspace
I've been trying to activate serverless clusters in Azure Databricks, but I can't find the feature enabled in the workspace console settings. My understanding is that this feature should be available in all regions; I'm currently working in US East.…
asked 2024-12-20T15:08:30.3533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(208, 45%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAM%3C/text%3E%3C/svg%3E)
Alejandro Montenegro Ortiz
40
Reputation points
commented 2024-12-20T17:53:00.8566667+00:00
Alejandro Montenegro Ortiz
40
Reputation points
1 answer
Need to create a policy which allows specific users for cluster creation in azure databricks
Need help to create a policy which allows specific users for cluster creation in azure databricks
asked 2024-11-28T12:47:02.7133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(105.60000000000001, 11%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESN%3C/text%3E%3C/svg%3E)
Sahitya Nigam
0
Reputation points
commented 2024-12-20T17:46:02.11+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(256, 20%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPR%3C/text%3E%3C/svg%3E)
Pranay Reddy Madireddy
1,230
Reputation points Microsoft Vendor
2 answers
Not Able To Access DataBricks Manage Account Console. Logging through global admin account. Can anyone help what to be done to resolve this.
Getting above error when try to click on manage account in databricks. Can anyone guide me to resolve this issue.
asked 2024-12-18T19:09:28.6366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(64, 70%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERG%3C/text%3E%3C/svg%3E)
Rahul Goel
0
Reputation points
commented 2024-12-20T16:06:45.87+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 18%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGG%3C/text%3E%3C/svg%3E)
Ganesh Gurram
2,380
Reputation points Microsoft Vendor
1 answer
Read data from Databricks Delta Sharing to Azure Synapse
Hi, I am trying to read the data from Databricks Delta Sharing and write the data to Azure Synapse by referring this documentation https://docs.databricks.com/en/connect/external-systems/synapse-analytics.html#language-python I have constructed below…
asked 2024-12-18T15:05:16.13+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(313.6, 55.00000000000001%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECJ%3C/text%3E%3C/svg%3E)
Cheng, Jack
0
Reputation points
edited a comment 2024-12-20T16:05:49.97+00:00
Ganesh Gurram
2,380
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
not being able to create azure free account or even to upgrade to pay as u go
how am i suppose to learn the certification if am not being able to create account. i am really frustrated that am not being able to create account. provide me an efficient way to create account to practice for my certification
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(281.6, 81%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(67.2, 19%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESK%3C/text%3E%3C/svg%3E)
1 answer
I couldnt create Access tokens. I dont see the option of "user setting" in the profile
I couldnt create Access tokens. I dont see the option of "user setting". I have checked the "roles" under "Access control" where I have the "owner" role. I have checked the Workspace admin settings,I could see I…
asked 2024-12-18T10:01:48.6+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(163.2, 65%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJY%3C/text%3E%3C/svg%3E)
JANARDH Y
0
Reputation points
commented 2024-12-20T11:06:52.79+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
18,625
Reputation points Microsoft Vendor
1 answer
I could see that In the 'Roles' in Access Control, Im the owner of my account. But I was not able to create accesss control. I think I dont have the permissions of a Databricks workspace admin
In azure databricks, I wanted to create Access Token. But Im not able to. I may not have the permissions of a Databricks workspace admin. How do i do that
asked 2024-12-18T05:11:23.9333333+00:00
JANARDH Y
0
Reputation points
commented 2024-12-20T11:05:03.4866667+00:00
Smaran Thoomu
18,625
Reputation points Microsoft Vendor
1 answer
can the path of source files be the same as the volume's location path?
hi friend, I currently have multiple source files stored on SA (Storage Account) in a directory called /raw/, which also contains a subdirectory /raw/sub1/. When I create a volume, the location is also pointing to SA. Can this location path point to…
asked 2024-12-20T03:33:25.1933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(92.8, 46%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHW%3C/text%3E%3C/svg%3E)
Huang, Winston-HR
300
Reputation points
answered 2024-12-20T04:18:48.9666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(304, 6%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKD%3C/text%3E%3C/svg%3E)
Keshavulu Dasari
2,410
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
How to get utilization(cpu,memory) report of Adb cluster
I have three clusters in adb i need its utilization report how can i get that
asked 2024-12-19T16:01:18.6566667+00:00
Aetemaad Hussain
20
Reputation points
edited a comment 2024-12-19T17:33:17.9+00:00
Ganesh Gurram
2,380
Reputation points Microsoft Vendor
1 answer
How to configure default Lakehouse in notebook in Fabric
I want to configure defualt laekhouse in notebook. i can do it using %%configure command. but i am note able to use variables here to be used in %%configure command. As configure should be the first cell of the notebook and if you put it in middle of the…
asked 2024-12-16T06:53:15.4766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(28.799999999999997, 15%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKB%3C/text%3E%3C/svg%3E)
Ketan Bharat | MAQ Software
60
Reputation points
commented 2024-12-18T02:01:09.1+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
5,755
Reputation points Microsoft Vendor
1 answer
Concurrency issue in partitioned table while updating same table from different spark notebook
Created partitioned table with below schema CREATE TABLE Landing.TargetTable ( Id INT, Name STRING, SourceSystem INT, SourceId INT, ETLId INT ) USING DELTA PARTITIONED BY (SourceId); Two source tables are there from where data are…
asked 2024-12-10T11:23:16.31+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 52%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHD%3C/text%3E%3C/svg%3E)
heta desai
357
Reputation points
commented 2024-12-17T01:16:02.7466667+00:00
Chandra Boorla
5,755
Reputation points Microsoft Vendor
1 answer
Can a Databricks workspace use an access connector to retrieve secrets from Key Vault?
faced error when trying to access secrets by adding the access connector object ID as key Vault Administrator on Key Vault.
asked 2024-12-10T15:32:28+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(256, 17%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMD%3C/text%3E%3C/svg%3E)
Mitasha Dutta (LTIMINDTREE LIMITED)
0
Reputation points Microsoft Vendor
commented 2024-12-16T13:32:53.8866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(236.8, 89%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHE%3C/text%3E%3C/svg%3E)
Harshitha Eligeti
1,205
Reputation points Microsoft Vendor
2 answers
One of the answers was accepted by the question author.
Issue with logging MLflow run in Databricks - BAD_REQUEST: Unable to connect to the linked AzureML workspace
I am encountering an issue while trying to log MLflow runs to an experiment and register a model in Azure Databricks. While executing the mlflow.start_run(), I continually receive a RestException: BAD_REQUEST error with the message "Unable to…
asked 2023-11-07T08:20:43.3633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 84%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJ%3C/text%3E%3C/svg%3E)
Joe
25
Reputation points
answered 2024-12-16T09:00:56.7833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(89.60000000000001, 32%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENM%3C/text%3E%3C/svg%3E)
Narsingh Maurya
0
Reputation points
1 answer
How to configure IP access list to databricks workspace
How to configure the ip access list to databricks workspace. I have seen the microsoft link for this : https://learn.microsoft.com/en-us/azure/databricks/security/network/front-end/ip-access-list-workspace but not found (databricks workspace-conf…
asked 2024-12-11T15:51:42.62+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(291.2, 96%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ER%3C/text%3E%3C/svg%3E)
rkreddy
0
Reputation points
commented 2024-12-16T01:54:13.32+00:00
Chandra Boorla
5,755
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Analysis of Cost Saved
Hi, Current Solution: Delta table from HiveMetastore -> Staging Table with Business Logic in Databricks -> PowerBI Current Solution in Detailed: Main responsibilities for me is to consume the data the delta tables from hive_metastore in…
asked 2024-12-11T05:47:30.34+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(137.6, 43%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJA%3C/text%3E%3C/svg%3E)
Jayaswaroop Adabala
20
Reputation points
accepted 2024-12-13T06:50:15.1566667+00:00
Jayaswaroop Adabala
20
Reputation points
3 answers
Selected user account does not exist in tenant 'Microsoft Services' and cannot access the application
Following Establish your first account admin bellow: https://learn.microsoft.com/en-us/azure/databricks/admin/#account-admins I login with owner, global admin account in azure portal, and Go to accounts.azuredatabricks.net and sign in with my global…
asked 2024-12-09T21:33:07.3366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(16, 70%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EXZ%3C/text%3E%3C/svg%3E)
Xuanbin Zhang
0
Reputation points
commented 2024-12-13T03:58:23.1366667+00:00
Chandra Boorla
5,755
Reputation points Microsoft Vendor
2 answers
Unable to create compute with single node 14GB memory, 4 core for azure databricks with azure student account
Hi Microsoft Team, I am using an Azure student account and I am facing issues when creating the compute for Azure Databricks. I have given a single node and then I selected 14GB of memory and 4 core General purpose as driver type and when I clicked on…
asked 2024-12-10T06:59:29.19+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(211.20000000000002, 12%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EYM%3C/text%3E%3C/svg%3E)
Yashwanth Maringanti
0
Reputation points
commented 2024-12-12T13:32:36.73+00:00
Ganesh Gurram
2,380
Reputation points Microsoft Vendor
2 answers
One of the answers was accepted by the question author.
Seeking Expertise in Spark SQL CTE Recursive Queries in Azure Databricks
I'm currently diving deep into Spark SQL and its capabilities, and I'm facing an interesting challenge. I'm eager to learn how to write CTE recursive queries in Spark SQL, but after thorough research, it seems that Spark doesn't natively support…
asked 2024-05-30T04:27:35.4+00:00
AnujSinghCognizant-4026
90
Reputation points
accepted 2024-12-12T13:20:35.1633333+00:00
AnujSinghCognizant-4026
90
Reputation points