Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,312 questions
0 answers
Facing challenges with Azure databricks cluster spin up
We have opened the firewalls to west us IP addresses and domains for Azure Databricks services and assets - Azure Databricks | Microsoft Learn but still we are not able to spin the cluster the cluster spin up fails with ailure message: Bootstrap script…
asked 2025-01-23T18:44:19.6666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 85%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERS%3C/text%3E%3C/svg%3E)
Rashmi Sunkeneni
0
Reputation points
commented 2025-01-27T10:52:55.9766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
19,310
Reputation points Microsoft Vendor
1 answer
How to configure default Lakehouse in notebook in Fabric
I want to configure defualt laekhouse in notebook. i can do it using %%configure command. but i am note able to use variables here to be used in %%configure command. As configure should be the first cell of the notebook and if you put it in middle of the…
asked 2024-12-16T06:53:15.4766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(28.799999999999997, 15%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKB%3C/text%3E%3C/svg%3E)
Ketan Bharat | MAQ Software
60
Reputation points
edited the question 2025-01-27T05:06:50.6233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(153.6, 16%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERG%3C/text%3E%3C/svg%3E)
Rakesh Gurram
11,475
Reputation points Microsoft Vendor
0 answers
Serverless not showing for my notebooks in Databricks
Hi. Serverless option is not showing for my notebooks and I cannot even see it in the feature enablement in accounts administration. In the attached screenshots, you can see that I have unity catalog enable and that serverless does not appear in feature…
asked 2025-01-26T12:10:14.4433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(128, 39%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMA%3C/text%3E%3C/svg%3E)
Mohammed Amine Mostefai
0
Reputation points
edited the question 2025-01-27T04:17:19.88+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 76%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EV%3C/text%3E%3C/svg%3E)
VarunTha
11,555
Reputation points Microsoft Vendor
1 answer
Unable to access unity catalogue
Unable to access managed account console, unable to access.azuredatabricks.net its showing that "Selected user account does not exist in tenant 'Microsoft Services' and cannot access the application '2ff814a6-3304-4ab8-85cb-cd0e6f879c1d' in that…
asked 2025-01-22T12:23:45.4933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 55.00000000000001%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAR%3C/text%3E%3C/svg%3E)
Akshay Raut
0
Reputation points
commented 2025-01-24T16:35:20.8133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
7,260
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Retrieving files created by Databricks Jobs
I am currently working with a Databricks workspace through Azure and using the Databricks Jobs API to execute Python scripts. These scripts create files during their execution. I would like to understand how to retrieve these files after the job…
asked 2025-01-21T13:46:50.5133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 15%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EG2%3C/text%3E%3C/svg%3E)
Gabriel-2005
425
Reputation points
accepted 2025-01-24T16:29:19.2566667+00:00
Gabriel-2005
425
Reputation points
1 answer
Retrieving 'Person or Group' Fields from SharePoint List in Azure Data Factory or Synapse
I'm able to retrieve data from a SharePoint list using Azure Data Factory > Copy Data (via the SharePoint Connector), but the list contains 'Person or Group' type fields that return only 'Id' values in the results. These 'Id' values are not useful,…
asked 2024-11-12T14:14:03.32+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 82%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESP%3C/text%3E%3C/svg%3E)
Sunny Punalkar
0
Reputation points
commented 2025-01-24T10:06:51.8133333+00:00
Smaran Thoomu
19,310
Reputation points Microsoft Vendor
1 answer
Delta Tables in Unity Catalog - My Organization vs Unity Catalog - Legacy - Hive Metastore
We have multiple Databricks Delta Tables that were created before Unity Catalog came into picture. After Unity Catalog got included in our Project, all our Tables are shown in Unity Catalog as part of Legacy. We would like to know if having them as…
asked 2025-01-22T20:07:40.7633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(240, 59%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EIN%3C/text%3E%3C/svg%3E)
Immaneni, Nanda
0
Reputation points
commented 2025-01-24T07:35:43.2933333+00:00
Chandra Boorla
7,260
Reputation points Microsoft Vendor
1 answer
Unable to Access Databricks Serverless Compute from CLI and VS Code
The following conditions exist while using Databricks with the premium plan: Unity catalog is enabled. No complaint security profile is enabled. The workspace is located in a supported region for Serverless Notebook compute — West US2. The…
asked 2025-01-21T15:57:00.0333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(6.4, 71%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EIH%3C/text%3E%3C/svg%3E)
Isaac Huang
5
Reputation points
commented 2025-01-23T21:51:22.5433333+00:00
Chandra Boorla
7,260
Reputation points Microsoft Vendor
2 answers
Unable to login to Azure Databricks Account Console
I have a personal Azure pay-as-you-go subscription in which I have the 'Global Administrator' role. I am also the databricks account administrator. Since two weeks I am not able anymore to access the databricks account console.
asked 2024-08-19T10:23:32.4833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(211.20000000000002, 37%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMW%3C/text%3E%3C/svg%3E)
Markus Weiler
10
Reputation points
commented 2025-01-23T09:32:29.9733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(227.2, 54%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHV%3C/text%3E%3C/svg%3E)
Hemanth Vasi
0
Reputation points
1 answer
Unable to remove resource group, and deny assigments, in Azure.
In azure I cant remove a resource group, and I cant remove deny assigments. I think it could be like this example: https://stackoverflow.com/questions/60694149/unable-to-remove-azure-databricks-managed-resource-group
asked 2025-01-22T16:22:59.9533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(217.60000000000002, 73%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDH%3C/text%3E%3C/svg%3E)
Dan Heikenberg
0
Reputation points
edited the question 2025-01-23T05:14:32.3566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(176, 78%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPM%3C/text%3E%3C/svg%3E)
Prasad-MSFT
8,071
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Connecting Databricks through Microsoft Purview by enabling "No Public Access"
Hi, I was able to connect Databricks through Microsoft Purview using Public Network but now the requirement is to connect Databricks through Microsoft Purview by enabling "No Public Access" in Databricks. The Databricks account is created…
asked 2023-04-01T05:36:28.74+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(92.8, 93%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPY%3C/text%3E%3C/svg%3E)
Paarveandan Y K
60
Reputation points
commented 2025-01-22T12:41:40.7066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(105.60000000000001, 1%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDP%3C/text%3E%3C/svg%3E)
Dhananjay Punekar
0
Reputation points
1 answer
In Synapse Notebook using pyspark inbuilt functions how to convert country code to country name as description
Hi Team, Could you please help us that how to change country code to country name (as description) for a column in synapse notebook using pyspark (modules or libraries) could you please share the code. we have used below but it is not working. …
asked 2025-01-17T09:13:38.3733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(163.2, 22%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESM%3C/text%3E%3C/svg%3E)
SaiSekhar, MahasivaRavi (Philadelphia)
60
Reputation points
commented 2025-01-21T09:32:59.79+00:00
Smaran Thoomu
19,310
Reputation points Microsoft Vendor
1 answer
Why am i Getting this error even if i am having subscription?
Azure Quota Exceeded Exception: Hitting Azure quota limits: Error code: QuotaExceeded, error message: Operation could not be completed as it results in exceeding approved standardDADSv5Family Cores quota. Additional details - Deployment Model: Resource…
asked 2025-01-16T13:57:44.34+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 22%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERA%3C/text%3E%3C/svg%3E)
Rangari, Ankit Shripatrao
0
Reputation points
commented 2025-01-20T06:45:08.78+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
13,150
Reputation points Microsoft Vendor
2 answers
��������� � in databricks
Hi Expert, how to replace below wild character from databricks ��������� �
asked 2024-02-02T11:43:29.5233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(140.8, 75%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESR%3C/text%3E%3C/svg%3E)
Shambhu Rai
1,411
Reputation points
answered 2025-01-20T04:18:13.15+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(105.60000000000001, 80%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EES%3C/text%3E%3C/svg%3E)
Evanjo Souza
0
Reputation points
4 answers
Using Microsoft Azure Free Trial for Databricks
I want to try out Databricks using my free trial. I am able to create a workspace, but when I try to create and run a cluster within it I get an error saying I have exceeded my core quota for the free trial. Is there a way I can try out Databricks…
asked 2020-09-10T16:02:49.623+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(211.20000000000002, 81%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ED%3C/text%3E%3C/svg%3E)
dfhgffg
1
Reputation point
commented 2025-01-19T08:32:22.45+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(112.00000000000001, 34%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESA%3C/text%3E%3C/svg%3E)
Sachin Agrawal
5
Reputation points
1 answer
One of the answers was accepted by the question author.
Data factory Copy activity fails when copy the data unity catalog table from databricks to storage account gen2
Data factory Copy activity fails when copy the data unity catalog table from databricks to storage account gen2 How to fix the connectivity issue even after providing all required access on catalog schema and tables level. error details: …
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 19%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EV%3C/text%3E%3C/svg%3E)
1 answer
One of the answers was accepted by the question author.
Integrate storage, database, webapp, and databricks into one VNET
We are hardening our ML platform in Azure. Basically, we want to put all the Azure services ( ADLS, PostgreSQL DB, Web App, and Databricks cluster) in one virtural network so all the traffic will be inside the Azure network whenever possible (except the…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(262.40000000000003, 27%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPJ%3C/text%3E%3C/svg%3E)
1 answer
unity catalog issue, i build 2 databricks workspace, but can not manage accounts
I created two workspaces in Azure Databricks, but I cannot find the 'Manage Accounts' option in my account. How can I access this service?" I can not use unity catalog service , any action I can fix this issue? thank you
asked 2025-01-13T17:19:54.9666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(153.6, 79%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
莎琦 陳
0
Reputation points
commented 2025-01-16T20:46:45.63+00:00
Chandra Boorla
7,260
Reputation points Microsoft Vendor
0 answers
How to resolve New-AzResourceGroupDeployment: - The deployment 'DataBricksDeployment' failed with error(s). Showing 1 out of 1 error(s). Status Message: The definition of application 'Databricks25' cannot be null. (Code:InvalidApplicationDefi
God day Does anyone know what could be causing this error. New-AzResourceGroupDeployment: 10:06:59 - The deployment 'DataBricksDeployment' failed with error(s). Showing 1 out of 1 error(s). Status Message: The definition of application…
asked 2025-01-14T08:32:04.5133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(102.4, 51%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETM%3C/text%3E%3C/svg%3E)
Tsholofelo Mautla
21
Reputation points
commented 2025-01-16T16:29:40.5133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 36%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
Bhargava-MSFT
31,196
Reputation points Microsoft Employee
0 answers
Lineage was extracted from unity but duplicate process created for same table In Purview
Hi Microsoft Team, I have registered and scanned databricks unity catalog in azure purview and then all tables,views metadata came into purview and that time lineage is not pulled into azure purview and now i have enabled access schema in unity catalog…
asked 2025-01-07T12:52:16.9333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 5%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESL%3C/text%3E%3C/svg%3E)
Sri Lakshman Velugubantla
20
Reputation points
commented 2025-01-16T11:21:17.3933333+00:00
Smaran Thoomu
19,310
Reputation points Microsoft Vendor