Data ingestion from azure function to ADX

Question

hi all,

I am trying to get an azure function app going to write the incoming http json payload to ADX table.

below are the steps i followed:

1. Create a new DB in ADX and create a corresponding table.
2. create a new function app using node and create a function which receives incoming payload over HTTP.
3. Provide function app Ingestor permission over DB in ADX.
4. Create a connection using system managed identity and new client with Azure-kusto-data package
   1. const kustoConnectionString = KustoConnectionStringBuilder.withSystemManagedIdentity(Cluster); const kustoClient = new Client(kustoConnectionString);
5. create a custom payload json matching Table structure, and create an ingestion query.
   1. const ingestionQuery = `.ingest inline into table ${Table} <| ${JSON.stringify(payload)}`;
6. Try execute this query using the client in step 4

kustoClient.execute(Database, ingestionQuery)

        .then(result => {

            context.log("Query Result:", result);

        })

        .catch(err => {

            context.log("Error executing query:", err);

            throw new Error('Error processing Request: ' + err.message);

        });

    } 

The function app runs fine with no errors but data never lands in ADX DB table. i tried checked .ignestion logs in DB but there is nothing. The ADX is currently configured to receive data from all IP addresses. Can someone advise what am i missing here ?

Answer 1

Hello @Abhishake Jaiswal

welcome to this moderated Azure community forum.

I tested this with C# and see my finding in this blog post.

I hope it give you pointers on how to write data into the database.

It also discusses direct and queued calls.

That said, I recommend to use an EventHub in between the Azure Function and Azure Data Exploxer.

Just output your messages to the EventHub and use a data connection to ingest the data.

This makes the flow more flexible and configurable (due to the data connection settings).

You can also look into the stream using the eventhub tooling.

---

If the response helped, do "Accept Answer". If it doesn't work, please let us know the progress. All community members with similar issues will benefit by doing so. Your contribution is highly appreciated.