Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,239 questions
0 answers
cannot find data files for Exercise - Integrate a Notebook within Azure Synapse Pipelines
I want to follow the workshop at https://learn.microsoft.com/en-us/training/modules/orchestrate-data-movement-transformation-azure-data-factory/5a-exercise-integrate-notebook-azure-synapse-pipelines However, the exercise starts with data loading and I…
asked 2025-03-14T21:11:40.1733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(316.8, 98%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAK%3C/text%3E%3C/svg%3E)
Antonina Korepanova
0
Reputation points
asked 2025-03-14T21:11:40.1733333+00:00
Antonina Korepanova
0
Reputation points
1 answer
One of the answers was accepted by the question author.
Can't create a new azure synapse analytics resource: No location is accepted for my subscription
I keep on getting this error when i try to create an azure synapse analytics resource. I tried multiple different regions but still the same error. Maybe something is wrong with my subscription. This is the error message i get:…
asked 2025-03-14T12:45:18.0133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(307.2, 84%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EEK%3C/text%3E%3C/svg%3E)
Elena Kovacheska
0
Reputation points
accepted 2025-03-14T16:59:36.14+00:00
Elena Kovacheska
0
Reputation points
1 answer
One of the answers was accepted by the question author.
Does azure synapse analytics support using custom VNet?
Hi there, Does azure synapse analytics support using custom VNet? What services on Azure support the use of custom virtual networks? Thanks&Regards, zmsoft
asked 2025-02-05T05:22:45.1533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(64, 53%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EZ%3C/text%3E%3C/svg%3E)
zmsoft
365
Reputation points
commented 2025-03-14T16:49:33.7333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
9,995
Reputation points Microsoft External Staff
1 answer
Convert Dataverse Table to JSON String in Synapse Pipeline
I am working on a pipeline in Azure Synapse Analytics, and I need to convert a Dataverse table to a JSON string. I want to know if it's possible to achieve this using a Web Activity within the pipeline. If so, could someone please explain how it can be…
asked 2025-03-14T12:16:31.21+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(6.4, 21%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAP%3C/text%3E%3C/svg%3E)
Ananya patil
20
Reputation points
answered 2025-03-14T13:34:23.6533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 18%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGG%3C/text%3E%3C/svg%3E)
Ganesh Gurram
5,045
Reputation points Microsoft External Staff
0 answers
Investigating and solving UserErrorSqlDWCopyCommandError error
Hi, I'm managing some ADF pipelines to write data on a Synapse Analytics dedicated SQL pool. Sometimes, not always, some pipelines finish with a timeout error. In particular, for an ADF pipeline I've the UserErrorSqlDWCopyCommandError error, with this…
asked 2025-03-14T10:23:17.2766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 41%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
pmscorca
1,032
Reputation points
edited a comment 2025-03-14T11:55:47.0366667+00:00
Ganesh Gurram
5,045
Reputation points Microsoft External Staff
2 answers
Update/Transform Headerless CSV Source
I am attempting to create a new dataflow to sink into a SQL table from a CSV source. My csv is missing it's header row though, so I'm stumbling on how to map the columns. It is delimited via pipe ('|'). Previous answers I have found reference a Headers…
asked 2025-03-10T23:28:23.6066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 33%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENF%3C/text%3E%3C/svg%3E)
Nestor Fregoso
0
Reputation points
answered 2025-03-14T08:55:24.05+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(6.4, 57.00000000000001%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERG%3C/text%3E%3C/svg%3E)
Rakesh Govindula
175
Reputation points Microsoft External Staff
0 answers
Recommended CI/CD Setup for Azure Synapse
Most tutorials and recommendations for how to set up CICD in azure synapse suggest creating multiple workspaces (e.g. -dev, -qa, -uat, -prod) and using a devops pipeline to move artifacts between the environments. Including this video tutorial from the…
asked 2025-03-11T09:28:23.9766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(272, 28.999999999999996%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EFS%3C/text%3E%3C/svg%3E)
Finn Schmidt
86
Reputation points
commented 2025-03-13T22:28:02.86+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(252.8, 99%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJN%3C/text%3E%3C/svg%3E)
J N S S Kasyap
860
Reputation points Microsoft External Staff
0 answers
Azure Synapse Link for Dataverse - How to delete audit entries after a specified period of time
We have a Synapse Link for Dataverse that sends audit table data into Synapse. When setting up the Synapse Link for Dataverse, Audit and Audit_Partitioned tables were automatically created. This setup works well, and all audit entries appear as…
asked 2025-03-13T12:14:43.24+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(102.4, 38%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EUA%3C/text%3E%3C/svg%3E)
Umar Ali
0
Reputation points
commented 2025-03-13T18:12:09.37+00:00
Chandra Boorla
9,995
Reputation points Microsoft External Staff
1 answer
How to access secrets from Key Vault from Synapse Spark Job Definition PySpark file
Hello Everyone I'm trying to read the data source credentials as a secrets from the Key Vault using the Python SDK on a PySpark Job. I've gone through many articles and used different ways to read the secrets from Key Vault, but nothing worked out.…
asked 2025-03-04T07:32:09.6066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(83.2, 22%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EM%3C/text%3E%3C/svg%3E)
manigandan
0
Reputation points
commented 2025-03-13T12:52:50.1933333+00:00
manigandan
0
Reputation points
1 answer
Restoring synapselink parquet files
Hi. We accidentally unlinked our audit table in synapselink for dataverse. Now we lost all data stored by synapselink from over 6 months and have been deleting data from d365. Is there anyway to recover the parquet files automatically created by…
asked 2025-03-13T07:44:51.5533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(28.799999999999997, 7.000000000000001%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJH%3C/text%3E%3C/svg%3E)
Jani Hämäläinen
105
Reputation points
answered 2025-03-13T11:01:20.86+00:00
Amira Bedhiafi
29,791
Reputation points
3 answers
I am trying to use the default sql storage of synapse ws as the sink for my data, but it throws logon failure
Here is the error I am getting Cannot connect to SQL database: 'jdbc:sqlserver://<synapse workspace name>.sql.azuresynapse.net:1433;database={Built-in}', 'Managed Identity (factory name): <synapse workspace name>'.[SQL Exception]Error…
asked 2025-02-07T12:14:52.76+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(140.8, 26%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESN%3C/text%3E%3C/svg%3E)
Seema Nair
0
Reputation points
commented 2025-03-13T05:02:23.1133333+00:00
Ganesh Gurram
5,045
Reputation points Microsoft External Staff
1 answer
Azure Synapse Link Roles
Hi Team I have a Synapse Link With F&O now we are transitioning to Production, but all the UAT process has been made with SysAdmin permissions on the Power Platform Environment, I tried to find documentation about the roles but can't find anything,…
asked 2025-03-10T18:50:37.9866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 99%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMA%3C/text%3E%3C/svg%3E)
Manuel Alejandro Estrada Doñes
20
Reputation points
commented 2025-03-13T01:03:09.57+00:00
Ganesh Gurram
5,045
Reputation points Microsoft External Staff
1 answer
One of the answers was accepted by the question author.
Is It Possible to Parse JSON Data in an External Table with Serverless SQL Pool?
Hello, I’m currently working on creating an external table in Azure Synapse Serverless SQL Pool using a JSON Lines formatted file. However, I’ve encountered a limitation: Serverless SQL Pool does not natively support JSON file formats for external table…
asked 2025-03-11T20:37:10.5866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 98%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESF%3C/text%3E%3C/svg%3E)
Saul Figueroa
40
Reputation points
accepted 2025-03-12T21:03:22.7033333+00:00
Saul Figueroa
40
Reputation points
1 answer
Deterministic encryption algorithm for encrypting a data column
Hi, I need to encrypt only a table column of data possibly with a deterministic algorithm and a fixed key. I've tried to use the simple simmetric encryption, but for each execution changing the related encrypted column. Any suggests to me, please? Thanks
asked 2024-03-03T17:51:27.79+00:00
pmscorca
1,032
Reputation points
commented 2025-03-12T16:42:08.53+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 18%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVS%3C/text%3E%3C/svg%3E)
Vasili Seniuts
0
Reputation points
1 answer
One of the answers was accepted by the question author.
What’s the best approach to create external tables from JSON logs using Serverless SQL Pool?.
Hello, I am currently using Azure Logging to monitor logs in real-time via Log Stream. The goal is to store these logs in a Storage Account Gen 2 container and create an external table in Azure Synapse Analytics for data analysis. However, I…
asked 2025-03-10T21:16:18.3+00:00
Saul Figueroa
40
Reputation points
accepted 2025-03-12T13:12:00.8933333+00:00
Saul Figueroa
40
Reputation points
1 answer
How to remove/unset Spark configs that are pre-set by Microsoft
Background Synapse Apache Spark configurations allow users to create a blueprint for a Spark configuration that can be used for customizing the Spark behavior. This blueprint can be assigned to a Spark Pool (and will be used as new default for new…
asked 2025-03-08T12:09:21.8333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 42%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMB%3C/text%3E%3C/svg%3E)
Martin B
126
Reputation points
commented 2025-03-12T09:12:30.75+00:00
Rakesh Govindula
175
Reputation points Microsoft External Staff
2 answers
Unable to update datetime to sink via data flow
Hi, I want to pass in a parameter to the data flow. This variable is current UTC value. @formatDateTime(utcNow(), 'yyyy-MM-dd HH:mm:ss.fff') In my data flow, I have a derived column. My sink column is timestamp data…
asked 2025-03-05T06:01:45.7+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(89.60000000000001, 14.000000000000002%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EL%3C/text%3E%3C/svg%3E)
Lotus88
116
Reputation points
commented 2025-03-11T20:22:54+00:00
Chandra Boorla
9,995
Reputation points Microsoft External Staff
1 answer
Unable to read content of file despite able to see FileInfo. urlopen error [Errno 5] Input/output error
I have 2 notebooks in different accounts, Staging & Production. Both use Managed Identity, linkedService -> System-assigned managed identity, and mounted drive. Both notebooks use the exact same code: Both can see the FileInfo, name, size,…
asked 2025-03-07T05:28:18.5866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(67.2, 85%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDW%3C/text%3E%3C/svg%3E)
Daniel Wang
5
Reputation points
commented 2025-03-11T17:57:46.8366667+00:00
J N S S Kasyap
860
Reputation points Microsoft External Staff
1 answer
One of the answers was accepted by the question author.
Synapselink Audit table source files
Hi. We have Synapselink for dataverse that send audit table data. The link creates audit and audit_partitioned tables automatically. Because of need to update spark pool there was pause in automatic table syncing. Now Audit_partitioned is missing some…
asked 2025-03-07T10:45:39.65+00:00
Jani Hämäläinen
105
Reputation points
commented 2025-03-11T15:55:24.2966667+00:00
Vinodh247
29,781
Reputation points MVP
3 answers
Connect Local Machine to Synapse Lake Database using Python
Hi, I am trying to develop tooling for my use case, where I am using PySpark/Spark SQL on Synapse Analytics to create tables in Lake database. Sample code below: spark.sql("CREATE TABLE abc AS SELECT * FROM view") The table abc is created in…
asked 2025-02-17T15:02:43.2566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(310.4, 88%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHN%3C/text%3E%3C/svg%3E)
Haria, Neel
0
Reputation points
commented 2025-03-11T09:14:40.2633333+00:00
AnnuKumari-MSFT
34,351
Reputation points Microsoft Employee