How to build a dashboard that displays the KPIs of sensor data in real-time on Azure

Question

I connected sensor data to the IoT hub, and it's connected to the time series insights. So the data flow is like this: sensor data-> IoT hub -> time series insights
The sensor message is a JSON message that contains temperature, humidity, and pressure values. I have a KPI formula. I want to feed the formula with this sensor's data and calculate the result in real-time. The result should be visualized as a report on Power BI. Then, I want to embed the report on our website. So, it would be great if the dashboard be in HTML format.
I'm considering connecting the IoT hub to the Cosmos DB. Then, use a Stream Analytics job to run queries on the database (input: CosmosDB), and calculate the KPI and send the result into Power BI (output: PowerBI).

1. I don't know how I can create and publish a real-time report of Power BI on the web. (I could make a real-time dashboard on Power BI. But, I can just share it with team members). I tried to connect the azure stream database to Power BI to create a report. But the report doesn't show the data stream in real-time (it is not auto-refresh).
2. I'm also curious about alternative solutions. Can I, for example, define a query on time series insights to calculate KPIs in real-time and send calculated value to Power BI?

I would appreciate it if you could suggest the best architecture in this case.

Thanks in advance,
Best regards

Answer 1

@Pezhman Sharafdin Welcome to Microsoft Q&A forum!

I don't know how I can create and publish a real-time report of Power BI on the web. (I could make a real-time dashboard on Power BI. But, I can just share it with team members). I tried to connect the azure stream database to Power BI to create a report. But the report doesn't show the data stream in real-time (it is not auto-refresh).

Please note that, You can add Power BI as an output within Azure Stream Analytics (ASA), and then visualize those data streams in the Power BI service in real time.

Azure Stream Analytics uses the Power BI REST APIs to create its output data stream to Power BI, with defaultMode set to pushStreaming, resulting in a dataset that can take advantage of both push and streaming. When the dataset is created, Azure Stream Analytics sets the retentionPolicy flag to basicFIFO. With that setting, the database supporting its push dataset stores 200,000 rows, and which rows are dropped in a first-in first-out (FIFO) fashion.

If your Azure Stream Analytics query results in very rapid output to Power BI (for example, once or twice per second), Azure Stream Analytics will begin batching those outputs into a single request. This may cause the request size to exceed the streaming tile limit. In that case, as mentioned in previous sections, streaming tiles will fail to render. In such cases, the best practice is to slow the rate of data output to Power BI; for example, instead of a maximum value every second, set it to a maximum over 10 seconds.

Take a look at this doc Visualize real-time sensor data from your Azure IoT hub in a web application and let us know if it helps. Below is the architecture involved in Visualize real-time sensor data from your Azure IoT hub in a web application.

I'm also curious about alternative solutions. Can I, for example, define a query on time series insights to calculate KPIs in real-time and send calculated value to Power BI?

Azure Time Series Insights Gen2 enables data querying on events and metadata stored in the environment via public surface APIs. These APIs also are used by the Azure Time Series Insights TSI Explorer. See Querying Data from Azure Time Series Insights Gen2 for more details.

!Note: The Time Series Insights (TSI) service will no longer be supported after March 2025. Consider migrating existing TSI environments to alternative solutions as soon as possible. For more information on the deprecation and migration, visit our documentation.

Do let us know if you have any further queries. We would be happy to help you.

If the response is helpful, please click "Accept Answer" and upvote it.

Answer 2

Check out the Azure version:

• https://learn.microsoft.com/en-us/azure/data-explorer/azure-data-explorer-dashboards

Or the Fabric Real Time Intelligence version:

• https://learn.microsoft.com/en-us/fabric/real-time-intelligence/dashboard-real-time-create
• Also check out https://learn.microsoft.com/en-us/fabric/real-time-intelligence/dashboard-explore-data
• More functionality included here like dashboard discovery, organizing dashboards in Fabric workspace/folder and more is coming.
• And more is coming.

To continue using the specific TSI controls you can still use Kusto (Azure Data Explorer or Fabric Eventhouse) and use the TSI controls hosted in a simpoe web app:

• https://techcommunity.microsoft.com/t5/azure-data-explorer-blog/kusto-trender/ba-p/3651241

All these options work seamlessly both with Azure Data Explorer timeseries databases or with Microsoft Fabric Eventhouse/KQL databases (Both are the same data service underneath: Kusto)