Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,364 questions
0 answers
Connecting Spark to Azure SQL
Hi, I'm following this article to connect Spark to Azure SQL https://learn.microsoft.com/en-us/azure/azure-sql/database/spark-connector?view=azuresql&source=docs It says that I need to get the Spark connector and it has a link to a maven repo that…
asked 2025-03-12T10:06:58.9866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(188.79999999999998, 36%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EYB%3C/text%3E%3C/svg%3E)
Yaish, Batel
0
Reputation points
commented 2025-03-12T19:04:51.5+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(252.8, 99%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJN%3C/text%3E%3C/svg%3E)
J N S S Kasyap
865
Reputation points Microsoft External Staff
1 answer
How can I make sure that a table and csv file can merge in an azure databricks query?
I am currently trying to copy data from a csv file into my table in azure databricks and am getting the following error. [DELTA FAILED TO MERGE FIELDS] Failed to merge fields ‘transfer_month’ and ‘transfer_month’ I am unable to solve the error. I can…
asked 2025-03-09T03:42:05.1566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(48, 49%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKK%3C/text%3E%3C/svg%3E)
Kohdai Kurihara
20
Reputation points
commented 2025-03-12T08:05:53.2966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 18%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGG%3C/text%3E%3C/svg%3E)
Ganesh Gurram
4,965
Reputation points Microsoft External Staff
1 answer
How do I use the azure databricks dlt pipeline to consume azure Event Hub data
Hi there, How do I use the azure databricks dlt pipeline to consume azure Event Hub data? EH_NAME = "myeventhub" TOPIC = "myeventhub" KAFKA_BROKER = "{EH_NAMESPACE}.servicebus.windows.net:9093" GROUP_ID =…
asked 2025-03-10T03:52:50.48+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(64, 53%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EZ%3C/text%3E%3C/svg%3E)
zmsoft
360
Reputation points
edited a comment 2025-03-12T04:30:28.01+00:00
Ganesh Gurram
4,965
Reputation points Microsoft External Staff
2 answers
One of the answers was accepted by the question author.
Need to connect to IMB DB2 from Azure databricks workspace
Need to connect to IMB DB2 (DB version IBM DB2 for i 7.3) form azure databricks workspace, the DB2 is hosted is AS400 server and DB port 8471. Please suggest on the step by step procedure to connect to DB and query a table, like installing required…
asked 2023-08-22T04:58:34.3733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(265.6, 96%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVR%3C/text%3E%3C/svg%3E)
veerabhadra reddy kovvuri
181
Reputation points
commented 2025-03-11T21:34:52.1633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 33%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGR%3C/text%3E%3C/svg%3E)
Gummadavelli, Raghu
0
Reputation points
1 answer
How to create a Materialized View in Azure Databricks stored in a Managed Catalog, based on data loaded from a foreign catalog
I am currently facing a problem with creating a materialized view in Catalog_2.Schema_B (Managed_Catalog) based on data I have connected from postgre SQL external source loaded into Catalog1_Schema_A (Foreign_Catalog). The error I get only says…
asked 2025-03-07T05:03:46.0333333+00:00
Kohdai Kurihara
20
Reputation points
commented 2025-03-11T20:28:43.39+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
9,985
Reputation points Microsoft External Staff
1 answer
Azure Databricks
We have our production Azure Databricks resources configured with two subnets: one for the host and one for the container, both using a /27 CIDR block. As we plan to execute multiple jobs and only have 16 IPs remaining, we have decided to expand the…
asked 2025-03-11T13:19:59.02+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 34%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHK%3C/text%3E%3C/svg%3E)
Hemanth Kumar
0
Reputation points
commented 2025-03-11T13:40:11.0066667+00:00
Hemanth Kumar
0
Reputation points
1 answer
manage account in azure databricks
Hi Guys hope you are fine. I am trying to login to https://accounts.azuredatabricks.net/login with my intra id and giving it same password of my azure portal acount but it is giving me the error that password is not correct. although it is working fine…
asked 2025-03-06T09:57:37.7066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 86%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMA%3C/text%3E%3C/svg%3E)
Muhammad Awais Javed
20
Reputation points
commented 2025-03-11T11:06:49.63+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
21,325
Reputation points Microsoft External Staff
2 answers
One of the answers was accepted by the question author.
How do I add users from different domains to azure databricks
Hi there, How do I add users from different domains to azure databricks? Thanks&Regards, zmsoft
2 answers
How to become azure databricks expert?
How to become azure databricks expert? Any Microsoft offered courses apart from microsoft learn
asked 2025-03-08T22:15:26.7966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 9%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGD%3C/text%3E%3C/svg%3E)
Goddeti Dhanasekhara Reddy
20
Reputation points
edited the question 2025-03-11T07:50:44.5466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(217.60000000000002, 24%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAA%3C/text%3E%3C/svg%3E)
Abiola Akinbade
24,090
Reputation points
2 answers
Users in a specific region are experiencing connection issues to Databricks workspaces
We have users that are experiencing issues with connecting to both our production and development environments of databricks through our Azure subscription. Everything else in Azure is working fine for them, though. We have other users in different…
asked 2025-03-03T18:21:41.94+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(275.2, 47%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKA%3C/text%3E%3C/svg%3E)
Kiel Auer
0
Reputation points
commented 2025-03-10T15:15:55.0433333+00:00
Chandra Boorla
9,985
Reputation points Microsoft External Staff
3 answers
No Support for reading zip files store in Azure Datalake storage using abfss protocol url via python notebook.
Hi, This is so strange that microsoft have created the abfss protocol but have zero support to read zip files using abfss protocol via python notebook. For example I have this sample code that gives me hard time identifying a zip file as a zip…
asked 2025-03-03T11:08:21.7866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 56.00000000000001%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJ%3C/text%3E%3C/svg%3E)
Jeevesh
0
Reputation points
commented 2025-03-10T08:41:17.19+00:00
J N S S Kasyap
865
Reputation points Microsoft External Staff
1 answer
What are the differences between notebooks (written in SQL) and queries in Azure Databricks?
What are the differences between notebooks (written in SQL) and queries in Azure Databricks? I am particularly interested in what the differences are in terms of performance, billing and maintenance. If there are no significant functional differences, is…
asked 2025-03-06T01:01:02.6+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(208, 50%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPB%3C/text%3E%3C/svg%3E)
Power BI Test User 2
0
Reputation points
commented 2025-03-10T05:31:02.3866667+00:00
Ganesh Gurram
4,965
Reputation points Microsoft External Staff
4 answers
One of the answers was accepted by the question author.
How to disable Azure Databricks Personal Compute policy?
Hello, can you please provide details how to disable Azure Databricks Personal Compute policy for a) all users b) non-admin users Thanks!
asked 2023-04-24T14:17:34.23+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 83%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAA%3C/text%3E%3C/svg%3E)
Anita A
110
Reputation points
answered 2025-03-07T14:48:58.82+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(256, 57.99999999999999%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERG%3C/text%3E%3C/svg%3E)
Robert Golik (o365)
1
Reputation point
1 answer
Databricks Cluster Deployment Error: BOOTSTRAP_TIMEOUT (SERVICE_FAULT)
Assistance is needed with an issue encountered while deploying a cluster in Databricks UAT. The following error message is received: run failed with error message cluster 305161200-quoa9grg was terminated. Reason: BOOTSTRAP_TIMEOUT (SERVICE_FAULT)
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(156.8, 2%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENP%3C/text%3E%3C/svg%3E)
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
0 answers
Azure Databricks Workspace will not open and generates a blank webpage. Console says Error 429
Access to the workspace was fine last Friday, but today the webpage fails to load. I've cleared my cache, tried multiple browsers, restarted my computer and still face the same error. When I open the console in the browser I see a host of 429 errors (Too…
asked 2025-03-03T14:18:38.67+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(172.8, 24%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDR%3C/text%3E%3C/svg%3E)
Devin Robbins
0
Reputation points
commented 2025-03-06T16:48:37.68+00:00
J N S S Kasyap
865
Reputation points Microsoft External Staff
1 answer
How can I enable the 'SHAREPOINT' type connection in Azure Databricks?
I tried to connect SharePoint using Azure Databricks Lakehouse Federation. First, I select Connection type "Microsoft SharePoint" to create a Lakehouse Federation. Next, I input authentication info and sign in with Microsoft SharePoint. Then,…
asked 2025-03-05T02:35:40.4366667+00:00
Power BI Test User 2
0
Reputation points
answered 2025-03-06T15:44:16.3133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(6.4, 57.00000000000001%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERG%3C/text%3E%3C/svg%3E)
Rakesh Govindula
95
Reputation points Microsoft External Staff
1 answer
Creating an Azure Databricks service but get Admin error when I'm an Owner?
Hi Forum, I have an Azure account and I want to do some work with Azure Databricks using my personal email to set it up. I created a pay as you go Azure Databricks workspace in Azure and all is good until I click on Launch. I get a note saying I should…
asked 2025-03-01T21:31:05.39+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(89.60000000000001, 1%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPB%3C/text%3E%3C/svg%3E)
Peter Boothby
1
Reputation point
commented 2025-03-06T02:39:32.98+00:00
Ganesh Gurram
4,965
Reputation points Microsoft External Staff
1 answer
How can I prohibit ordinary users to execute notebooks of shared folder in Azure Databricks?
Hello, How can I prohibit ordinary users to create, modify, and execute notebooks of shared folder in Azure Databricks? For security reasons, I want to prevent ordinary users from accidentally sharing the results of SELECT sensitive data in the Shared…
asked 2025-02-25T04:03:49.2733333+00:00
Power BI Test User 2
0
Reputation points
commented 2025-03-06T00:57:41.2566667+00:00
J N S S Kasyap
865
Reputation points Microsoft External Staff
0 answers
How to read zip file in Azure Synapse Notebook with extension of .zip.
Hi Team, Could you please advise the pyspark code, how to read zip files in synapse notebook. Example: File_Name - samplefile.zip Note: In the samplefile.zip we have multiple csv files, please let us know. The file Extension is only with .zip. …
asked 2024-11-08T16:44:52.83+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(163.2, 22%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESM%3C/text%3E%3C/svg%3E)
SaiSekhar, MahasivaRavi (Philadelphia)
100
Reputation points
commented 2025-03-05T07:39:18.4533333+00:00
Jeevesh
0
Reputation points
1 answer
One of the answers was accepted by the question author.
Is there any way to export/download queries in order to move queries from one subscription to another??
I have been using Azure Databricks on one subscription and have queries in a workspace. Now I want to move them to another databricks workspace in a different subscription. Is there any way for me to export or download queries? My current solution is to…
asked 2025-03-03T07:53:14.71+00:00
Kohdai Kurihara
20
Reputation points
accepted 2025-03-05T02:54:40.0166667+00:00
Kohdai Kurihara
20
Reputation points