How to recover data for failed ingestion

Question

Hello ADX experts.

I'm using queued ingestion which seems to always be successful (i.e. inserted to queue) regardless of whether the queued data can be merged to the main table. (Please correct me if I'm wrong.)

I did a quick test and temporarily renamed a table. The queued ingestions were still succesful, but the batches could not be merged to the table (obvisouly), and appeared when running the .show ingestion failures commmand.

However, I can't find a way to recover those data. Is this possible? I tried opening the IngestionSourcePath URL, but it only results in an error.

If it's not possible, do I have any options for where failed ingestions could be stored? For example a poison queue or similar?

This is an important feature for us, as we need to be able to reliably store the data when the queue command has been successful.

Answer 1

Hi Pål Kristian Halle,

Thanks for reaching out to Microsoft Q&A.

The full ingestion process can be monitored using ingestion metrics and diagnostic logs. Ingestion failures can be monitored using the IngestionResult metric or the FailedIngestion diagnostic log. The .show ingestion failures command shows ingestion failures associated with the data ingestion management commands, and isn't recommended for monitoring errors. The .dup-next-failed-ingest command provides information on the next failed ingestion by uploading ingestion files and metadata to a storage container. This can be useful for checking an ingestion flow, though isn't advised for steady monitoring.

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

Answer 2

Hello @Pål Kristian Halle,

welcome to this moderated Azure community forum.

Have you checked the KustoIngestionProperties offered by the SDK?

KustoQueuedIngestionProperties class extends KustoIngestionProperties with several control knobs that can be used to fine-tune the ingestion behavior.

You can set the report level:

ingestionProperties.ReportLevel = IngestionReportLevel.FailuresAndSuccesses;    ingestionProperties.ReportMethod = IngestionReportMethod.Queue;

This way you have more control over the ingestion.

Notice that this has a huge impact on ingestion performance so use this with care! It is recommended to use this reporting feature under diagnostics circumstances only.

---

If the response helped, do "Accept Answer". If it doesn't work, please let us know the progress. All community members with similar issues will benefit by doing so. Your contribution is highly appreciated.