Separate data in data explorer and use as datastore

Question

Hello,

We are sending data from IoT Central to Event Hubs and then to Data Explorer, with the hopes of then sending the data to Azure Machine Learning.

In order to send data from Event Hubs to Data Explorer it needs a data ingestion into a table on data explorer.

For this data ingestion, it needs a json mapping.

We could ingest the data, but the message from the iot central data goes to event hubs that goes to data explorer carries the telemetry data as a dynamic type (a json inside a json).

("telemetry":{"Temp:"37","Vol":"97"})  

We want to separate the telemetry data in different columns.

So Temp will have one column and Vol another.

I am wondering how that can be done?

And additionally, since we would like to send the data to ML, can data explorer be used as a datastore in ML?

Thanks!!

Answer 1

Hello @yjay ,

You can use parse operator - Evaluates a string expression and parses its value into one or more calculated columns. The calculated columns will have nulls, for unsuccessfully parsed strings.

For more details, refer SO thread addressing similar issue.

Unfortuantely, Azure Data Explorer is not a supported storage solution with Azure Machine Learning.

Datastores currently support storing connection information to the storage services listed in the following matrix.

For unsupported storage solutions, and to save data egress cost during ML experiments, move your data to a supported Azure storage solution.

Reference: Connect to storage services on Azure - Azure Machine Learning.

Hope this helps. Do let us know if you any further queries.

------------

Please don’t forget to Accept Answer and Up-Vote wherever the information provided helps you, this can be beneficial to other community members.