This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(307.2, 31%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERS%3C/text%3E%3C/svg%3E)
I am following this documentation to send logs and metrics for Spark applications running on Synapse Spark pools to Event Hub using Azure Key Vault (AKV).
I am using the configuration below and have added the Service Principal secret to AKV. I also tried putting the connection string in the AKV secrets, but I receive an error:
Your notebook connection has closed unexpectedly. To re-establish the connection, run the notebook again. Diagnostic information: kernel_connection_closed (70az88)
spark.synapse.diagnostic.emitters MyDestination1
spark.synapse.diagnostic.emitter.MyDestination1.type AzureEventHub
spark.synapse.diagnostic.emitter.MyDestination1.categories Log,EventLog,Metrics
spark.synapse.diagnostic.emitter.MyDestination1.secret.keyVault <...>
spark.synapse.diagnostic.emitter.MyDestination1.secret.keyVault.secretName <...>
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Apologies for delay in response.
Greetings & Welcome to Microsoft Q&A forum! Thanks for posting your query!
It seems that you are encountering issues with the Spark Diagnostic Emitter when trying to use Azure Key Vault for storing your connection string. One known issue is that storing secrets in Key Vault can prevent Spark sessions from starting, which may be causing the error you are experiencing.
To resolve this, it is recommended to configure the Spark properties using the connection string directly instead of relying on Azure Key Vault. Ensure that you have the correct permissions set for the Service Principal to read secrets from the Key Vault if you choose to continue using it.
Make sure your configuration file is set up correctly, including the necessary parameters for the Event Hub and that the connection string is formatted properly.
For more details, please refer: Get an Event Hubs connection string
I hope this information helps. Please do let us know if you have any further queries.
Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues.
Thank you.
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution, please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution, please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues.
Thank you.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(272, 19%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERS%3C/text%3E%3C/svg%3E)
This will help you answer your question:
https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/azure-synapse-diagnostic-emitters-azure-eventhub?wt.mc_id=studentamb_422941
I am following this link only but looks like it is not working when using AKV based creds