Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,228 questions
0 answers
How to remove/unset Spark configs that are pre-set by Microsoft
Background Synapse Apache Spark configurations allow users to create a blueprint for a Spark configuration that can be used for customizing the Spark behavior. This blueprint can be assigned to a Spark Pool (and will be used as new default for new…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 42%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMB%3C/text%3E%3C/svg%3E)
1 answer
Azure Synapse Analytics is not getting registered when working with Lab " Explore Azure Synapse Analytics"
getting error while trying the lab Explore Azure Synapse Analytics Registering resource providers... Microsoft.Synapse is not yet registered. Waiting for 30 seconds before rechecking... Microsoft.Synapse is not yet registered. Waiting for 30 seconds…
asked 2024-12-14T10:38:47.2766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(144, 66%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMB%3C/text%3E%3C/svg%3E)
Monish Bhanushali
15
Reputation points
commented 2025-03-08T07:26:15.71+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(140.8, 86%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESD%3C/text%3E%3C/svg%3E)
Shubham Dubey
0
Reputation points
1 answer
Synapse Link Dataverse F&O Multiple Files Partitions
Hi Team We have an Azure Synapse Link Dataverse with F&O, with a partition by year and the option appendOnly checked. In all my tables I just getting one file per year that make me sense, but i have one table that for some reason is generating…
asked 2025-03-05T15:46:08.8833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 99%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMA%3C/text%3E%3C/svg%3E)
Manuel Alejandro Estrada Doñes
20
Reputation points
commented 2025-03-07T18:11:23.0033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
9,685
Reputation points Microsoft External Staff
2 answers
Unable to update datetime to sink via data flow
Hi, I want to pass in a parameter to the data flow. This variable is current UTC value. @formatDateTime(utcNow(), 'yyyy-MM-dd HH:mm:ss.fff') In my data flow, I have a derived column. My sink column is timestamp data…
asked 2025-03-05T06:01:45.7+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(89.60000000000001, 14.000000000000002%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EL%3C/text%3E%3C/svg%3E)
Lotus88
116
Reputation points
commented 2025-03-07T17:16:41.6433333+00:00
Chandra Boorla
9,685
Reputation points Microsoft External Staff
1 answer
Synapselink Audit table source files
Hi. We have Synapselink for dataverse that send audit table data. The link creates audit and audit_partitioned tables automatically. Because of need to update spark pool there was pause in automatic table syncing. Now Audit_partitioned is missing some…
asked 2025-03-07T10:45:39.65+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(28.799999999999997, 7.000000000000001%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJH%3C/text%3E%3C/svg%3E)
Jani Hämäläinen
80
Reputation points
edited the question 2025-03-07T15:53:09.4066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 76%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EV%3C/text%3E%3C/svg%3E)
VarunTha
12,950
Reputation points Microsoft External Staff
1 answer
How to access secrets from Key Vault from Synapse Spark Job Definition PySpark file
Hello Everyone I'm trying to read the data source credentials as a secrets from the Key Vault using the Python SDK on a PySpark Job. I've gone through many articles and used different ways to read the secrets from Key Vault, but nothing worked out.…
asked 2025-03-04T07:32:09.6066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(83.2, 22%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EM%3C/text%3E%3C/svg%3E)
manigandan
0
Reputation points
commented 2025-03-07T14:25:08.48+00:00
Chandra Boorla
9,685
Reputation points Microsoft External Staff
1 answer
Unable to read content of file despite able to see FileInfo. urlopen error [Errno 5] Input/output error
I have 2 notebooks in different accounts, Staging & Production. Both use Managed Identity, linkedService -> System-assigned managed identity, and mounted drive. Both notebooks use the exact same code: Both can see the FileInfo, name, size,…
asked 2025-03-07T05:28:18.5866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(67.2, 85%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDW%3C/text%3E%3C/svg%3E)
Daniel Wang
5
Reputation points
answered 2025-03-07T12:53:01.5766667+00:00
Amira Bedhiafi
29,711
Reputation points
0 answers
Unable to use Linked Service Connection Using "To File Path": The configuration is overwritten after Publishing and during usage
Configuring Access to Amazon S3 with Limited Permissions I am working with an Amazon S3 storage system where my access is restricted to a specific bucket/ directory, rather than having root-level access. This limited permission setup has led me to…
asked 2025-03-07T08:37:29.18+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 55.00000000000001%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
sr24
0
Reputation points
commented 2025-03-07T10:18:41.8033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
21,160
Reputation points Microsoft External Staff
1 answer
One of the answers was accepted by the question author.
Azure Synapse Apache Spark pool in Current state: DeleteError. How do I get it into a succeeded state?
Hi, I tried to delete an Apache Spark pool. Deletion failed. Jobs are still running, but I cannot make any changes to the configuration of the Spark pool. Neither can I delete it. I read that "unlinking all notebooks from the spark…
asked 2025-03-06T12:56:23.7466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(278.4, 33%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKB%3C/text%3E%3C/svg%3E)
Kajetan Bocek
40
Reputation points
accepted 2025-03-07T09:43:08.8333333+00:00
Kajetan Bocek
40
Reputation points
1 answer
Failed to update table. Change in table plan is limited to once a week. Last update was yesteday.
For a table in Log analytics workspace we had changed to Basic plan. We need it to be changed to Analytics immediately to reduce impact. But receiving error failed to update Change in table plan is limited to once a week. Last update was yesteday What…
asked 2025-02-28T07:36:02.4733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(0, 13%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES5%3C/text%3E%3C/svg%3E)
SirilJoseph-5276
0
Reputation points
commented 2025-03-07T04:20:17.6933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
14,390
Reputation points Microsoft External Staff
1 answer
One of the answers was accepted by the question author.
how to update apache spark pool version from3.3 to 3.4 if don't have any spark pools under the synapse workspace
i dont have spark pools configured under synapse analytics workspaces so how can i update syanpse spark pools to 3.4 version. is there any dependency validation required for this?
asked 2024-10-23T09:51:20.06+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(3.2, 98%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAG%3C/text%3E%3C/svg%3E)
ANKIT GHENGE
110
Reputation points
commented 2025-03-06T21:15:12.8666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(153.6, 89%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EOC%3C/text%3E%3C/svg%3E)
Ongweoweh Corp
0
Reputation points
2 answers
Synapse stuck in provisioning state
I deployed a Synapse Workspace a long time ago. It never finished deploying, and now I want to get rid of it, but I am unable to delete it. The deletion fails with message: "Workspace is in non-terminal state Provisioning and cannot be…
asked 2025-03-06T13:39:11.0166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(172.8, 28.000000000000004%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERK%3C/text%3E%3C/svg%3E)
Rasmus Kromann
0
Reputation points
answered 2025-03-06T20:58:39.2133333+00:00
Rasmus Kromann
0
Reputation points
1 answer
One of the answers was accepted by the question author.
Integrating Spark with Azure Synapse Dedicated sql instance
Hi, I am having a Synapse dedicate sql instance to which I am trying to read and write using Apache spark from Java application. But issue arise when trying to create a Table due to default Clustered index creation which does not allow MAX for Synapse…
asked 2025-03-04T07:01:47.7233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(294.40000000000003, 33%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENS%3C/text%3E%3C/svg%3E)
Nitish Sharma
40
Reputation points
accepted 2025-03-06T13:21:03.6866667+00:00
Nitish Sharma
40
Reputation points
0 answers
Issue with Synapse Lake Database Table
Hello, I am having an issue with Synapse Lake Database Table. I created a table in Lakehouse database, I was able to do a select and everything from pyspark notebook. However when i try to do the select from SSMS or synapse it doesn't work. It says table…
asked 2025-02-24T18:40:52.9966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 15%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAK%3C/text%3E%3C/svg%3E)
Admin (KK)
136
Reputation points
commented 2025-03-06T11:26:36.7333333+00:00
AnnuKumari-MSFT
34,271
Reputation points Microsoft Employee
0 answers
Why are incorrect v-cores allocated?
I created and ran a notebook with one executor. In the monitor section, i see that 48 cores are allocated and i am unable to utilize more cores in other notebooks. I have to wait almost 30 mins after i stop the spark session to create the session…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(224.00000000000003, 97%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMS%3C/text%3E%3C/svg%3E)
2 answers
Azure Data Factory - Copy data from AWS Redshift to Azure Synapse - Failed to connect error
Hi, I am trying to copy a table of data from AWS Redshift into Azure Synapse, below is the JSON code from the data factory: { "name": "pipeline1", "properties": { "activities": [ { …
asked 2025-02-18T22:50:02.35+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(89.60000000000001, 60%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVP%3C/text%3E%3C/svg%3E)
Vinny Prajapati
5
Reputation points
commented 2025-03-06T08:53:59.75+00:00
Vinny Prajapati
5
Reputation points
1 answer
One of the answers was accepted by the question author.
Failed to publish and activate the trigger
Hi, I built a pipeline in Synapse and added an event trigger on it so the pipeline will be executed once a new file is uploaded to my storage account. The issue is that I couldn't activate the trigger. When I started the trigger and clicked…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(54.400000000000006, 35%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAH%3C/text%3E%3C/svg%3E)
1 answer
Azure Synapse - slow performance issues
We are encountering slow performance issues with Azure Synapse. DWU is low yet queries are still taking too long to execute, even simple SELECT statements. Region is Southeast Asia. Issue was encountered around 1300H (GMT +8).
asked 2025-03-03T13:39:59.8233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 93%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ED%3C/text%3E%3C/svg%3E)
DWMA
0
Reputation points
commented 2025-03-05T12:02:09.3066667+00:00
AnnuKumari-MSFT
34,271
Reputation points Microsoft Employee
1 answer
How did Azure Synapse Analytics blow through so many credits?
I set up Azure Synapse Analytics a few days ago, but didn't complete the set up so hadn't even started using it (no data loaded). I logged back in today to finish setting it up and I noticed that it showed I had spent almost $5,000 over the weekend. I…
asked 2025-02-24T22:01:16.2233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 20%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chris Baker
0
Reputation points
answered 2025-03-05T10:25:36.0233333+00:00
AnnuKumari-MSFT
34,271
Reputation points Microsoft Employee
0 answers
How to read zip file in Azure Synapse Notebook with extension of .zip.
Hi Team, Could you please advise the pyspark code, how to read zip files in synapse notebook. Example: File_Name - samplefile.zip Note: In the samplefile.zip we have multiple csv files, please let us know. The file Extension is only with .zip. …
asked 2024-11-08T16:44:52.83+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(163.2, 22%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESM%3C/text%3E%3C/svg%3E)
SaiSekhar, MahasivaRavi (Philadelphia)
100
Reputation points
commented 2025-03-05T07:39:18.4533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 56.00000000000001%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJ%3C/text%3E%3C/svg%3E)
Jeevesh
0
Reputation points