Azure Queue Storage message timeout

Question

Hi everyone,

We're running a Javascript function in Azure Functions that links to Azure Queue Storage. When images come into the Blob Storage, the queue gets populated with messages and then the function uses "sharp" to process them further.

However, we're having issues where if I upload a batch of images, not every single one gets processed correctly.
I've tried stopping the Azure Function, uploading images and the queue shows the correct number of image messages. However, once I restart the function, it processes most of the images, but then queue messages will periodically pop back into the queue and a few images will always end up not completing.
Are there any Queue Storage settings I could look at to see what's causing this issue?
Sorry if I haven't explained my question well enough - it's tricky.

Kind regards
Chris.

Answer 1

Hi @Chris Buley ,

I understand that you are facing issues with Azure Queue Storage message timeout problem.

Please follow the below suggestions to resolve the issue.

• Adjust the visibility timeout to ensure the function has sufficient time to process each message. If the processing time surpasses the visibility timeout, the message will reappear in the queue.
• Adding retry logic can assist in managing transient errors and ensure messages are retried before being marked as failed.
• Please enable detailed logging to identify any specific errors or exceptions that occur during processing.
• Improve your function's performance to reduce processing time and ensure it handles exceptions smoothly.
• Test the function with different message loads and processing times to confirm it handles messages accurately.
• you can use Azure Monitor and Application Insights to monitor performance and pinpoint any bottlenecks or errors.

For additional information, please refer the following documents:

Setting timeouts for Queue service operations (REST API) - Azure Storage

Queue Storage REST API - Azure Storage | Microsoft Learn

I hope by following the above suggestions should be able to resolve the issue.

Please let us know in the comments section, if there are any further quires. We will be glad to assist you better.

Please do consider to “up-vote” wherever the information provided helps you, this can be beneficial to other community members.