Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,319 questions
1 answer
Serverless not showing for my notebooks in Databricks
Hi. Serverless option is not showing for my notebooks and I cannot even see it in the feature enablement in accounts administration. In the attached screenshots, you can see that I have unity catalog enable and that serverless does not appear in feature…
asked 2025-01-26T12:10:14.4433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(128, 39%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMA%3C/text%3E%3C/svg%3E)
Mohammed Amine Mostefai
0
Reputation points
commented 2025-02-01T17:32:33.0466667+00:00
Mohammed Amine Mostefai
0
Reputation points
0 answers
scim tenant url
I am new to azure and using a trial version, i want to connect azure to azure databricks for that i need scim tenant url, the thing is every tutorial video is saying tio go to ser management, but there is no user managment in the drop down in databricks,…
Azure
Azure
A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.
1,092 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,319 questions
asked 2025-02-01T16:21:36.3866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(137.6, 36%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESL%3C/text%3E%3C/svg%3E)
Saurabh Lokhande
0
Reputation points
asked 2025-02-01T16:21:36.3866667+00:00
Saurabh Lokhande
0
Reputation points
1 answer
How to connect from Azure Databricks to an prem MS SQL server
I have an Azure Databricks instance deployed on a Virtual Network. On the same (!) Virtual Network, I also have a Azure VM deployed, that is successfully able to connect to an on-prem MS SQL server. However, when I try to connect via the Azure…
asked 2025-01-30T17:17:51.58+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(137.6, 26%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EN%3C/text%3E%3C/svg%3E)
Nathalie
0
Reputation points
commented 2025-01-31T19:11:50.24+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
7,415
Reputation points Microsoft Vendor
2 answers
Using this mssparkutils.notebook.runMultiple function to run notebook in parallel, how to find the failed notebook.
Hi Team, Could you please help us to find the failed notebook in parallel run using the below code, but not able to find the failed one. Eg: If any notebook got failed it is stopping the processs, but we don't know which notebook failed , can you…
asked 2025-01-29T09:02:16.5633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(163.2, 22%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESM%3C/text%3E%3C/svg%3E)
SaiSekhar, MahasivaRavi (Philadelphia)
80
Reputation points
commented 2025-01-31T09:51:01.53+00:00
SaiSekhar, MahasivaRavi (Philadelphia)
80
Reputation points
1 answer
One of the answers was accepted by the question author.
Issue with Specifying Cluster Log Path During Databricks Unity Catalog Upgrade
Hi Team, As part of our upgrade to Unity Catalog, we are removing all mount point usages. During this process, we encountered an issue while specifying the cluster log destination in the Linked Service configuration. We noticed that DBFS paths (with…
asked 2025-01-28T13:09:23.2466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(38.4, 8%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESK%3C/text%3E%3C/svg%3E)
Sharukh Kundagol
145
Reputation points
accepted 2025-01-31T06:28:19.7066667+00:00
Sharukh Kundagol
145
Reputation points
0 answers
Not able to create azure databricks
Hi , I am not able to create azure databricks after upgrading to pay-as-you-go model as free tier does not have feature for unity catalog . Even if i give all the details for the mandatory fields still not able to create azure databricks. If anyone face…
asked 2025-01-30T02:56:30.52+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 95%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENS%3C/text%3E%3C/svg%3E)
Niranjan Swain
5
Reputation points
commented 2025-01-31T04:12:17.54+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 18%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGG%3C/text%3E%3C/svg%3E)
Ganesh Gurram
3,600
Reputation points Microsoft Vendor
1 answer
how to fix "quota exceed exception" for a free trail user subscription ?
Can anyone tell me which is the right configuration while Compute the cluster configuration. Need the info. worker type , min workers, max workers Driver type Additional input if it is mandatory. I selected the region East-US
asked 2025-01-28T07:50:02.03+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(236.8, 11%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EL%3C/text%3E%3C/svg%3E)
lukstudy
0
Reputation points
commented 2025-01-30T03:17:25.0666667+00:00
Ganesh Gurram
3,600
Reputation points Microsoft Vendor
1 answer
Retrieving 'Person or Group' Fields from SharePoint List in Azure Data Factory or Synapse
I'm able to retrieve data from a SharePoint list using Azure Data Factory > Copy Data (via the SharePoint Connector), but the list contains 'Person or Group' type fields that return only 'Id' values in the results. These 'Id' values are not useful,…
asked 2024-11-12T14:14:03.32+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 82%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESP%3C/text%3E%3C/svg%3E)
Sunny Punalkar
0
Reputation points
edited a comment 2025-01-29T09:17:03.84+00:00
Sunny Punalkar
0
Reputation points
1 answer
One of the answers was accepted by the question author.
In Synapse Notebook using pyspark inbuilt functions how to convert country code to country name as description
Hi Team, Could you please help us that how to change country code to country name (as description) for a column in synapse notebook using pyspark (modules or libraries) could you please share the code. we have used below but it is not working. …
asked 2025-01-17T09:13:38.3733333+00:00
SaiSekhar, MahasivaRavi (Philadelphia)
80
Reputation points
accepted 2025-01-29T08:55:31.1766667+00:00
SaiSekhar, MahasivaRavi (Philadelphia)
80
Reputation points
1 answer
DP-203 Labs
Is there any way to do all the labs for DP-203 without costing alot of money AND without taking like 10 minutes to install something? For each lab, the ./setup.ps1 file takes like 10 or 15 minutes to load. I'd like to not have to delete the resources…
asked 2024-07-21T00:03:12.49+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(291.2, 23%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBC%3C/text%3E%3C/svg%3E)
Ben Cohn
5
Reputation points
edited the question 2025-01-27T16:17:38.78+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(153.6, 16%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERG%3C/text%3E%3C/svg%3E)
Rakesh Gurram
11,630
Reputation points Microsoft Vendor
1 answer
How to configure default Lakehouse in notebook in Fabric
I want to configure defualt laekhouse in notebook. i can do it using %%configure command. but i am note able to use variables here to be used in %%configure command. As configure should be the first cell of the notebook and if you put it in middle of the…
asked 2024-12-16T06:53:15.4766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(28.799999999999997, 15%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKB%3C/text%3E%3C/svg%3E)
Ketan Bharat | MAQ Software
60
Reputation points
edited the question 2025-01-27T05:06:50.6233333+00:00
Rakesh Gurram
11,630
Reputation points Microsoft Vendor
1 answer
Unable to access unity catalogue
Unable to access managed account console, unable to access.azuredatabricks.net its showing that "Selected user account does not exist in tenant 'Microsoft Services' and cannot access the application '2ff814a6-3304-4ab8-85cb-cd0e6f879c1d' in that…
asked 2025-01-22T12:23:45.4933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 55.00000000000001%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAR%3C/text%3E%3C/svg%3E)
Akshay Raut
0
Reputation points
commented 2025-01-24T16:35:20.8133333+00:00
Chandra Boorla
7,415
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Retrieving files created by Databricks Jobs
I am currently working with a Databricks workspace through Azure and using the Databricks Jobs API to execute Python scripts. These scripts create files during their execution. I would like to understand how to retrieve these files after the job…
asked 2025-01-21T13:46:50.5133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 15%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EG2%3C/text%3E%3C/svg%3E)
Gabriel-2005
425
Reputation points
accepted 2025-01-24T16:29:19.2566667+00:00
Gabriel-2005
425
Reputation points
1 answer
Delta Tables in Unity Catalog - My Organization vs Unity Catalog - Legacy - Hive Metastore
We have multiple Databricks Delta Tables that were created before Unity Catalog came into picture. After Unity Catalog got included in our Project, all our Tables are shown in Unity Catalog as part of Legacy. We would like to know if having them as…
asked 2025-01-22T20:07:40.7633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(240, 59%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EIN%3C/text%3E%3C/svg%3E)
Immaneni, Nanda
0
Reputation points
commented 2025-01-24T07:35:43.2933333+00:00
Chandra Boorla
7,415
Reputation points Microsoft Vendor
1 answer
Unable to Access Databricks Serverless Compute from CLI and VS Code
The following conditions exist while using Databricks with the premium plan: Unity catalog is enabled. No complaint security profile is enabled. The workspace is located in a supported region for Serverless Notebook compute — West US2. The…
asked 2025-01-21T15:57:00.0333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(6.4, 71%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EIH%3C/text%3E%3C/svg%3E)
Isaac Huang
5
Reputation points
commented 2025-01-23T21:51:22.5433333+00:00
Chandra Boorla
7,415
Reputation points Microsoft Vendor
1 answer
Facing challenges with Azure databricks cluster spin up
We have opened the firewalls to west us IP addresses and domains for Azure Databricks services and assets - Azure Databricks | Microsoft Learn but still we are not able to spin the cluster the cluster spin up fails with ailure message: Bootstrap script…
asked 2025-01-23T18:44:19.6666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 85%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERS%3C/text%3E%3C/svg%3E)
Rashmi Sunkeneni
0
Reputation points
answered 2025-01-23T21:41:27.9033333+00:00
Chandra Boorla
7,415
Reputation points Microsoft Vendor
2 answers
Unable to login to Azure Databricks Account Console
I have a personal Azure pay-as-you-go subscription in which I have the 'Global Administrator' role. I am also the databricks account administrator. Since two weeks I am not able anymore to access the databricks account console.
asked 2024-08-19T10:23:32.4833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(211.20000000000002, 37%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMW%3C/text%3E%3C/svg%3E)
Markus Weiler
10
Reputation points
commented 2025-01-23T09:32:29.9733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(227.2, 54%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHV%3C/text%3E%3C/svg%3E)
Hemanth Vasi
0
Reputation points
1 answer
Unable to remove resource group, and deny assigments, in Azure.
In azure I cant remove a resource group, and I cant remove deny assigments. I think it could be like this example: https://stackoverflow.com/questions/60694149/unable-to-remove-azure-databricks-managed-resource-group
asked 2025-01-22T16:22:59.9533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(217.60000000000002, 73%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDH%3C/text%3E%3C/svg%3E)
Dan Heikenberg
0
Reputation points
edited the question 2025-01-23T05:14:32.3566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(176, 78%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPM%3C/text%3E%3C/svg%3E)
Prasad-MSFT
8,261
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Connecting Databricks through Microsoft Purview by enabling "No Public Access"
Hi, I was able to connect Databricks through Microsoft Purview using Public Network but now the requirement is to connect Databricks through Microsoft Purview by enabling "No Public Access" in Databricks. The Databricks account is created…
asked 2023-04-01T05:36:28.74+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(92.8, 93%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPY%3C/text%3E%3C/svg%3E)
Paarveandan Y K
60
Reputation points
commented 2025-01-22T12:41:40.7066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(105.60000000000001, 1%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDP%3C/text%3E%3C/svg%3E)
Dhananjay Punekar
0
Reputation points
1 answer
Why am i Getting this error even if i am having subscription?
Azure Quota Exceeded Exception: Hitting Azure quota limits: Error code: QuotaExceeded, error message: Operation could not be completed as it results in exceeding approved standardDADSv5Family Cores quota. Additional details - Deployment Model: Resource…
asked 2025-01-16T13:57:44.34+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 22%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERA%3C/text%3E%3C/svg%3E)
Rangari, Ankit Shripatrao
0
Reputation points
commented 2025-01-20T06:45:08.78+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
13,465
Reputation points Microsoft Vendor