Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,323 questions
1 answer
One of the answers was accepted by the question author.
How to check exact IP address of ADB workspace env .
i just want to know my IP address of Azure data bricks workspace env, I have vnet and subnet configured. How can i get exact IP address of ADB workspace. As i have more than 100 NIC to check the exact IP. Is their any easiest way to get exact ip…
asked 2025-02-04T02:27:14.43+00:00
Aetemaad Hussain
20
Reputation points
commented 2025-02-05T20:34:41.37+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 18%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGG%3C/text%3E%3C/svg%3E)
Ganesh Gurram
3,765
Reputation points Microsoft Vendor
1 answer
scim tenant url
I am new to azure and using a trial version, i want to connect azure to azure databricks for that i need scim tenant url, the thing is every tutorial video is saying tio go to ser management, but there is no user managment in the drop down in databricks,…
asked 2025-02-01T16:21:36.3866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(137.6, 36%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESL%3C/text%3E%3C/svg%3E)
Saurabh Lokhande
0
Reputation points
commented 2025-02-05T06:50:41.8033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
7,565
Reputation points Microsoft Vendor
2 answers
how to connect azure cache redis to azure databricks through service principal
I need to connect azure redis to databricks using service principal . but I am getting this error Failed to connect to Redis: invalid username-password pair
asked 2025-02-04T09:34:56.25+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(227.2, 80%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENP%3C/text%3E%3C/svg%3E)
Nageswara.Podapati
0
Reputation points
answered 2025-02-04T20:37:07.6833333+00:00
Oury Ba-MSFT
20,106
Reputation points Microsoft Employee
1 answer
Azure Databricks and Key vault key versioning
We are building a Data encryption solution and are using pyspark AES encrypt and AES Decrypt function to encrypt the PII columns in the Databricks tables on Unity catalog. We have created views on top of these tables and using the same encryption key we…
asked 2025-01-28T07:07:43.85+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 34%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHK%3C/text%3E%3C/svg%3E)
Hemanth Kumar
0
Reputation points
edited a comment 2025-02-04T07:20:09.2066667+00:00
Hemanth Kumar
0
Reputation points
1 answer
One of the answers was accepted by the question author.
Serverless not showing for my notebooks in Databricks
Hi. Serverless option is not showing for my notebooks and I cannot even see it in the feature enablement in accounts administration. In the attached screenshots, you can see that I have unity catalog enable and that serverless does not appear in feature…
asked 2025-01-26T12:10:14.4433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(128, 39%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMA%3C/text%3E%3C/svg%3E)
Mohammed Amine Mostefai
20
Reputation points
accepted 2025-02-03T15:53:01.5966667+00:00
Mohammed Amine Mostefai
20
Reputation points
1 answer
Unable to remove resource group, and deny assigments, in Azure.
In azure I cant remove a resource group, and I cant remove deny assigments. I think it could be like this example: https://stackoverflow.com/questions/60694149/unable-to-remove-azure-databricks-managed-resource-group
asked 2025-01-22T16:22:59.9533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(217.60000000000002, 73%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDH%3C/text%3E%3C/svg%3E)
Dan Heikenberg
0
Reputation points
edited the question 2025-02-03T13:55:02.9166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(281.6, 17%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGM%3C/text%3E%3C/svg%3E)
Givary-MSFT
35,216
Reputation points Microsoft Employee
1 answer
Lineage was extracted from unity but duplicate process created for same table In Purview
Hi Microsoft Team, I have registered and scanned databricks unity catalog in azure purview and then all tables,views metadata came into purview and that time lineage is not pulled into azure purview and now i have enabled access schema in unity catalog…
asked 2025-01-07T12:52:16.9333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 5%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESL%3C/text%3E%3C/svg%3E)
Sri Lakshman Velugubantla
20
Reputation points
answered 2025-02-03T12:24:43.1533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(83.2, 87%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPI%3C/text%3E%3C/svg%3E)
Prathista Ilango
170
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
We want to know details about azure local , can we use databricks , adlsg2, datafactory, machine learning on azure local ?
We want to know details about azure local , can we use databricks , adlsg2, datafactory, machine learning on azure local?
asked 2025-02-03T08:34:20.4466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(121.6, 68%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMM%3C/text%3E%3C/svg%3E)
Mohammad Mahamudul Hasan
20
Reputation points
accepted 2025-02-03T10:09:22.4133333+00:00
Mohammad Mahamudul Hasan
20
Reputation points
1 answer
One of the answers was accepted by the question author.
How to connect from Azure Databricks to an prem MS SQL server
I have an Azure Databricks instance deployed on a Virtual Network. On the same (!) Virtual Network, I also have a Azure VM deployed, that is successfully able to connect to an on-prem MS SQL server. However, when I try to connect via the Azure…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(137.6, 26%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EN%3C/text%3E%3C/svg%3E)
3 answers
ADF Copy Data JSON Source dynamic schema mapping
Hi I am working on ADF Copy data activity. HTTP Dataset is returning a JSON with the following sample JSON output { "totalRowCount": 1, "data": [ { "ProductCode": "P - 1", …
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(22.400000000000002, 44%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EM%3C/text%3E%3C/svg%3E)
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 52%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EV%3C/text%3E%3C/svg%3E)
1 answer
Not able to create azure databricks
Hi , I am not able to create azure databricks after upgrading to pay-as-you-go model as free tier does not have feature for unity catalog . Even if i give all the details for the mandatory fields still not able to create azure databricks. If anyone face…
asked 2025-01-30T02:56:30.52+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 95%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENS%3C/text%3E%3C/svg%3E)
Niranjan Swain
5
Reputation points
commented 2025-02-03T02:53:34.2233333+00:00
Ganesh Gurram
3,765
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Issue with Specifying Cluster Log Path During Databricks Unity Catalog Upgrade
Hi Team, As part of our upgrade to Unity Catalog, we are removing all mount point usages. During this process, we encountered an issue while specifying the cluster log destination in the Linked Service configuration. We noticed that DBFS paths (with…
asked 2025-01-28T13:09:23.2466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(38.4, 8%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESK%3C/text%3E%3C/svg%3E)
Sharukh Kundagol
145
Reputation points
accepted 2025-01-31T06:28:19.7066667+00:00
Sharukh Kundagol
145
Reputation points
1 answer
One of the answers was accepted by the question author.
Using this mssparkutils.notebook.runMultiple function to run notebook in parallel, how to find the failed notebook.
Hi Team, Could you please help us to find the failed notebook in parallel run using the below code, but not able to find the failed one. Eg: If any notebook got failed it is stopping the processs, but we don't know which notebook failed , can you…
asked 2025-01-29T09:02:16.5633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(163.2, 22%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESM%3C/text%3E%3C/svg%3E)
SaiSekhar, MahasivaRavi (Philadelphia)
100
Reputation points
commented 2025-01-30T13:52:19.5233333+00:00
SaiSekhar, MahasivaRavi (Philadelphia)
100
Reputation points
1 answer
how to fix "quota exceed exception" for a free trail user subscription ?
Can anyone tell me which is the right configuration while Compute the cluster configuration. Need the info. worker type , min workers, max workers Driver type Additional input if it is mandatory. I selected the region East-US
asked 2025-01-28T07:50:02.03+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(236.8, 11%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EL%3C/text%3E%3C/svg%3E)
lukstudy
0
Reputation points
commented 2025-01-30T03:17:25.0666667+00:00
Ganesh Gurram
3,765
Reputation points Microsoft Vendor
1 answer
Retrieving 'Person or Group' Fields from SharePoint List in Azure Data Factory or Synapse
I'm able to retrieve data from a SharePoint list using Azure Data Factory > Copy Data (via the SharePoint Connector), but the list contains 'Person or Group' type fields that return only 'Id' values in the results. These 'Id' values are not useful,…
asked 2024-11-12T14:14:03.32+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 82%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESP%3C/text%3E%3C/svg%3E)
Sunny Punalkar
0
Reputation points
edited a comment 2025-01-29T09:17:03.84+00:00
Sunny Punalkar
0
Reputation points
1 answer
One of the answers was accepted by the question author.
In Synapse Notebook using pyspark inbuilt functions how to convert country code to country name as description
Hi Team, Could you please help us that how to change country code to country name (as description) for a column in synapse notebook using pyspark (modules or libraries) could you please share the code. we have used below but it is not working. …
asked 2025-01-17T09:13:38.3733333+00:00
SaiSekhar, MahasivaRavi (Philadelphia)
100
Reputation points
accepted 2025-01-29T08:55:31.1766667+00:00
SaiSekhar, MahasivaRavi (Philadelphia)
100
Reputation points
1 answer
DP-203 Labs
Is there any way to do all the labs for DP-203 without costing alot of money AND without taking like 10 minutes to install something? For each lab, the ./setup.ps1 file takes like 10 or 15 minutes to load. I'd like to not have to delete the resources…
asked 2024-07-21T00:03:12.49+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(291.2, 23%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBC%3C/text%3E%3C/svg%3E)
Ben Cohn
5
Reputation points
edited the question 2025-01-27T16:17:38.78+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(153.6, 16%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERG%3C/text%3E%3C/svg%3E)
Rakesh Gurram
11,640
Reputation points Microsoft Vendor
1 answer
How to configure default Lakehouse in notebook in Fabric
I want to configure defualt laekhouse in notebook. i can do it using %%configure command. but i am note able to use variables here to be used in %%configure command. As configure should be the first cell of the notebook and if you put it in middle of the…
asked 2024-12-16T06:53:15.4766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(28.799999999999997, 15%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKB%3C/text%3E%3C/svg%3E)
Ketan Bharat | MAQ Software
60
Reputation points
edited the question 2025-01-27T05:06:50.6233333+00:00
Rakesh Gurram
11,640
Reputation points Microsoft Vendor
1 answer
Unable to access unity catalogue
Unable to access managed account console, unable to access.azuredatabricks.net its showing that "Selected user account does not exist in tenant 'Microsoft Services' and cannot access the application '2ff814a6-3304-4ab8-85cb-cd0e6f879c1d' in that…
asked 2025-01-22T12:23:45.4933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 55.00000000000001%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAR%3C/text%3E%3C/svg%3E)
Akshay Raut
0
Reputation points
commented 2025-01-24T16:35:20.8133333+00:00
Chandra Boorla
7,565
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Retrieving files created by Databricks Jobs
I am currently working with a Databricks workspace through Azure and using the Databricks Jobs API to execute Python scripts. These scripts create files during their execution. I would like to understand how to retrieve these files after the job…
asked 2025-01-21T13:46:50.5133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 15%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EG2%3C/text%3E%3C/svg%3E)
Gabriel-2005
445
Reputation points
accepted 2025-01-24T16:29:19.2566667+00:00
Gabriel-2005
445
Reputation points