IoT code sample - loading messages from Event Hub into Azure SQL Database
Paolo Salvatori created an example that simulates an Internet of Things (IoT) scenario where thousands of devices send events (e.g. sensor readings) to a backend system via a message broker. The backend system retrieves events from the messaging infrastructure and store them to a persistent repository in a scalable manner. Solution has the following components:
- Event Hub to collect messages
- Service Fabric to move messages from Event Hub into Azure SQL Database in JSON format
- Azure Sql Database where JSON messages are parsed using OPENJSON function and stored messages
The architecture is shown on the following figure:
Device simulator sends messages to Event Hub, and Service Fabric service reads messages as JSON and stores them into Azure SQL Database. Messages are sent in JSON format so they are parsed in database using OPENJSON function and stored into Events table.
Source code is available on GitHub:
https://github.com/azure-cat-emea/servicefabricjsonsqldb
Comments
- Anonymous
June 12, 2016
Which Visio template did you use for this diagram? Do you have a link? Thank you. - Anonymous
June 13, 2016
Hi MichaelI just invented my own style for Visio... ;)CiaoPaolo - Anonymous
June 14, 2016
This is Event Hub, not Service Fabric. The products and processing models are very different. I was about to ask "why use Service Fabric instead of an Event processor or Azure Function or Stream Analytics?".- Anonymous
June 14, 2016
Messages are initially stored in Even Hub, and Service Fabric is used just to transfer events from Event Hub to Azure SQL. In theory you can use Azure Data Factory, Azure Stream Analytics, Azure Functions or anything else to transfer data. Paolo used Service Fabric for this demo, maybe it would be better to post question on his page.I believe that using Azure Stream Analytics might be overkill if you need just to transfer data. ASA is used to stream data and analyze time series, so if you are using just streaming without analysis you are using just 10% of value. Azure Functions and web Jobs are not resilient to failures like Service Fabric. - Anonymous
September 28, 2016
The comment has been removed
- Anonymous