Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,527 questions
1 answer
Stream RealTime data from an On Prem Microsoft Server to a Azure Data Lake.
There is a need for Real Time data to be collected from a On Prem SQL Server. This means ADF is not suitable because it only does batch. Azure Streaming Analytics only connects to Azure SQL Server and not on prem. Has anybody else encountered this and…
asked 2023-03-16T02:02:54.12+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 50%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESW%3C/text%3E%3C/svg%3E)
Sallie Worsley
5
Reputation points
commented 2025-01-17T23:10:26.6+00:00
Sander van de Velde | MVP
34,281
Reputation points MVP
1 answer
Debugging Data Loss in Stream Analytics to Event Hubs and ADF
My goal is to stream telemetry data from blob storage files into a Azure Data Factory database. I have set up a stream analytics job that takes data from blob storage, bins it by hour, and then passes it through an event hub with two partitions in order…
asked 2025-01-16T03:01:19.43+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(64, 28.999999999999996%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAZ%3C/text%3E%3C/svg%3E)
Annie Zhou
0
Reputation points Microsoft Employee
commented 2025-01-17T17:31:50.76+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
19,140
Reputation points Microsoft Vendor
0 answers
We are seeing higher than normal latencies of around 4 seconds per request when sending events to Azure Event Hub. .
We send around 500k - 600k events through Azure Event Hub per day. Over the past week, we have seen our latencies for sending events to Event Hub increase to ~4 seconds per call. I thought that perhaps we were hitting some Throughput Units. But our Event…
asked 2025-01-16T17:54:08.8466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(48, 85%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESB%3C/text%3E%3C/svg%3E)
Steve Bering
0
Reputation points
commented 2025-01-17T16:37:35.75+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
6,780
Reputation points Microsoft Vendor
2 answers
Connect to eventhub using Managed service identity (MSI) from Azure synapse analytics using pyspark spark streaming
I am trying to write data to Azure Event Hubs from an Azure Synapse Analytics notebook in PySpark using Managed Service Identity (MSI). Could you please provide step-by-step guidance on how to achieve this, along with the necessary code? Please note, I…
asked 2024-09-26T05:45:48.2766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(70.4, 82%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESY%3C/text%3E%3C/svg%3E)
Sunil Yadav
0
Reputation points
edited a comment 2025-01-17T04:10:38.8433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 16%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDC%3C/text%3E%3C/svg%3E)
Deshpande, Chetan
0
Reputation points Microsoft Employee
2 answers
One of the answers was accepted by the question author.
im getting "unauthorized" status code 401 although i added the neccessary permissions
im trying to send logs from my storage to an event hub by using logic app. i added the permission "Azure Event Hubs Data Sender" to the logic app (managed identity). im getting "unauthorized" status code 401 on the last step, which is…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 80%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EG%3C/text%3E%3C/svg%3E)
0 answers
Cross-tenant data connection between event hub and ADX
HI Team, how to create Cross-tenant data connection between event hub and ADX using SPN authentication
asked 2025-01-09T02:42:53.4633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(112.00000000000001, 87%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERM%3C/text%3E%3C/svg%3E)
Raja Mohamed Masthan Syed Ahamed
0
Reputation points Microsoft Employee
commented 2025-01-13T04:19:53.8733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 18%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGG%3C/text%3E%3C/svg%3E)
Ganesh Gurram
3,110
Reputation points Microsoft Vendor
2 answers
How to monitor partition count currently in use for event hub namespaces?
We are trying to create an alert to notify us when our event hub namespaces are nearing partition count capacity. I have been able to find out how many partitions are allocated to an event hub namespace using KQL, but have not been able to figure out how…
asked 2025-01-06T20:35:55.2233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(316.8, 85%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAT%3C/text%3E%3C/svg%3E)
Aaron Turcott
0
Reputation points
commented 2025-01-09T23:57:26.9533333+00:00
Sander van de Velde | MVP
34,281
Reputation points MVP
1 answer
Azure Event Hub Geo-Disaster recovery
Your revised version looks great! Here's a slightly refined version with minor adjustments for flow: Hi, I am interested in implementing a geo-disaster recovery mechanism for Azure Event Hub. I have reviewed the guidelines regarding primary and…
asked 2025-01-07T02:16:33.1166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 94%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELL%3C/text%3E%3C/svg%3E)
Leonty Lahman
0
Reputation points
commented 2025-01-09T16:09:17.5666667+00:00
Smaran Thoomu
19,140
Reputation points Microsoft Vendor
1 answer
Azure Event Hubs Timeout Issue with High Throughput
Hello, a proof of concept is being conducted with Azure Event Hubs to ingest real-time data streams into storage. According to the documentation, a single Throughput Unit (TU) can handle up to 1,000 messages per second. However, in practice, with 10 TUs,…
asked 2025-01-03T10:51:46.28+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(233.6, 84%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKT%3C/text%3E%3C/svg%3E)
Koemuer, Taha Yusuf (148)
0
Reputation points
answered 2025-01-09T08:26:13.6266667+00:00
Koemuer, Taha Yusuf (148)
0
Reputation points
1 answer
How to Investigate Duplicate Event Processing in Event Hub
Events are being processed twice, resulting in duplicate notifications for both Email and In-App messages.
asked 2025-01-06T19:04:31.1933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(150.4, 61%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKU%3C/text%3E%3C/svg%3E)
Kunam, Umakanth
60
Reputation points
commented 2025-01-09T03:56:41.0966667+00:00
Ganesh Gurram
3,110
Reputation points Microsoft Vendor
1 answer
I have an application which sends table data from ibmias400 tables . How to collect messages and process it according to table names from messages and load it into delta parquet files/SQL tables and use it for reporting purposes?
I have an application which sends table data from ibmias400 tables . How to collect messages and process it according to table names from messages and load it into delta parquet files/SQL tables and use it for reporting purposes?
asked 2024-12-20T05:55:04.43+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(284.8, 25%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESS%3C/text%3E%3C/svg%3E)
Suraj Surendran
60
Reputation points
edited the question 2025-01-07T13:11:21.8533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(105.60000000000001, 79%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3END%3C/text%3E%3C/svg%3E)
Nikhil Duserla
3,960
Reputation points Microsoft Vendor
1 answer
Unable to connect azure eventhub emulator with Communication Exception: Connection refused: no further information
I try to connect the azure eventhub emulator using connection(Endpoint=sb://localhost;EntityPath=cpc-eh-aldp-forwarderdata-int-int01;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=SAS_KEY_VALUE;UseDevelopmentEmulator=true) string locally,…
asked 2025-01-02T06:39:30.4666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 9%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBX%3C/text%3E%3C/svg%3E)
Bingqiang Xiang
20
Reputation points Microsoft Employee
commented 2025-01-06T08:38:22.32+00:00
Smaran Thoomu
19,140
Reputation points Microsoft Vendor
2 answers
Having issue while streaming data from event hub into databricks using managed identity process
I'm trying to stream data from azure event hub to azure dataframe in databricks notebook using python. I have utilized managed Identity process to utilize passwordless process. It is giving the following error message when trying to stream the data.
asked 2024-12-31T15:08:36.46+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 62%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKS%3C/text%3E%3C/svg%3E)
Kallu, Srinath
0
Reputation points
answered 2025-01-02T17:32:49.4366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
13,060
Reputation points Microsoft Vendor
1 answer
Event Hub from another Subscription not visible for IoT Hub data route endpoints
Im trying to pass telemetry data from an IoT Hub to an event hub residing in another subscription, under same tenant/directory. I have created a managed identity, and assigned both services to the identity with Subscription-level Owner roles and…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(291.2, 4%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
3 answers
One of the answers was accepted by the question author.
Some questions with the Azure kafka service
we want use Azure kafka service, some questions need your help : Azure kafka is Confluent Cloud kafka? what's difference with Apache Kafka? Is support cross-region? What configuration is needed to support cross-region? Does it support native Kafka…
asked 2024-12-16T02:41:13.5+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(278.4, 84%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EYW%3C/text%3E%3C/svg%3E)
Yingping Wu
20
Reputation points
answered 2024-12-24T03:18:12.04+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(275.2, 9%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ET%3C/text%3E%3C/svg%3E)
Ts
0
Reputation points
1 answer
Inquiry Regarding Expiration of Azure Event Hub RootManageSharedAccessKey
Hi All, When I create an Azure Event Hub, a Shared Access Policy named RootManageSharedAccessKey is automatically created. I would like to confirm whether the access key within this policy has an expiration date, or if it is permanent Thank you.
asked 2024-12-15T09:42:08.2633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(313.6, 88%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EE%3C/text%3E%3C/svg%3E)
Eli
0
Reputation points
commented 2024-12-17T01:27:44.74+00:00
Chandra Boorla
6,780
Reputation points Microsoft Vendor
1 answer
unrecognized arguments: --eventhub-name
https://learn.microsoft.com/en-us/azure/event-hubs/event-hubs-get-connection-string#azure-cli has --namespace-name xxx in the instructions, but when I try it on my desktop or in cloud shell, I get this error: unrecognized arguments: --eventhub-name xxx
asked 2024-12-10T02:25:12.8266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 24%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKK%3C/text%3E%3C/svg%3E)
Klinemeier, Kevin (624)
0
Reputation points
commented 2024-12-16T19:38:51.7433333+00:00
phemanth
13,060
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
IoT Hub Continues to Send Events After Device Deletion
Hi Everyone, I have an issue with IoT Hub, Could you help ? Events such as TwinChangeEvents and DeviceLifecycleEvents are routed from IoT Hub to Event Hub. However, after deleting the deviceId (S5_U0_15312) created in 2016, IoT Hub still sends…
asked 2024-12-11T08:29:35.52+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(150.4, 27%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETT%3C/text%3E%3C/svg%3E)
Tuan Tran
21
Reputation points
commented 2024-12-12T19:40:02.2666667+00:00
Sander van de Velde | MVP
34,281
Reputation points MVP
1 answer
One of the answers was accepted by the question author.
View Events shows error - At least one receiver for the endpoint is created with epoch of '6'.
What is the meaning of this error and how can I resolve it? I get this error when I use "View Events" option in the "Data Explorer" in the Event Hub instance. "At least one receiver for the endpoint is created with epoch of '6',…
asked 2024-12-11T02:22:46.5933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(227.2, 57.00000000000001%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPD%3C/text%3E%3C/svg%3E)
Poonam Deshmukh
40
Reputation points
accepted 2024-12-12T12:58:31.04+00:00
Poonam Deshmukh
40
Reputation points
1 answer
One of the answers was accepted by the question author.
Azure EvenHub integration with Elasticsearch
Hello, I want to send the events on the Event Hub or Azure Service Bus to Elastic (installed on an Azure VM). There are two integrations that I found - Azure Event Hub Integration and Azure Logstash Integration. Can you help me understand how to make a…
asked 2024-12-09T21:32:24.32+00:00
Poonam Deshmukh
40
Reputation points
commented 2024-12-11T23:03:18.4933333+00:00
Poonam Deshmukh
40
Reputation points