Azure Function App processing multiple queue messages at a time despite batchSize set to 1

Question

I have an Azure Function written in Python that's meant to process messages from an Azure Storage Queue. In my host.json file, I have set "batchSize": 1 and "newBatchThreshold": 0 under the queues settings. This should mean that the function dequeues and processes one message at a time. The function execution time will take minimum 1 minuets as per the business logic. This arrangement works just fine locally.

The function code

import azure.functions as func
import logging
import json
import os
import time

app = func.FunctionApp()

@app.queue_trigger(arg_name="msg", queue_name=os.getenv("REQUEST_QUEUE"), connection="AzureWebJobsStorage")
def ustorescraper(msg: func.QueueMessage):
    
    received_message = msg.get_body().decode('utf-8')
    json_data = json.loads(received_message)
    request_type_str = json_data["requestType"]

    logging.info('Process started : %s', request_type_str)
    time.sleep(65)
    logging.info('Processing end : %s', request_type_str)

host.json

{
  "version": "2.0",
  "logging": {
    "applicationInsights": {
      "samplingSettings": {
        "isEnabled": true,
        "excludedTypes": "Request"
      }
    },
    "logLevel": {
      "default": "Information"
    }
  },
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle",
    "version": "[4.*, 5.0.0)"
  },
  "extensions": {
    "queues": {
      "batchSize": 1,
      "newBatchThreshold": 0,
      "maxPollingInterval": "00:05:00",
      "visibilityTimeout": "00:00:30",
      "maxDequeueCount": 3,
      "messageEncoding": "base64"
    }
  }
}

However, when I deploy to Azure, the function seems to be processing multiple messages at a time, which is not what I want.

In an attempt to override the settings on Azure, I added "AzureFunctionsJobHost__extensions__queues__batchSize": "1" and "AzureFunctionsJobHost__extensions__queues__newBatchThreshold": "1" as environment variables to the Function App settings in the Azure portal, as per Microsoft's documentation.

Some of Environment variables opened as Advanced Edit

  {
    "name": "AzureFunctionsJobHost__extensions__queues__batchSize",
    "value": "1",
    "slotSetting": true
  },
  {
    "name": "AzureFunctionsJobHost__extensions__queues__newBatchThreshold",
    "value": "0",
    "slotSetting": true
  },
  {
    "name": "REQUEST_QUEUE",
    "value": "request-queue",
    "slotSetting": false
  },
  {
    "name": "AzureWebJobsStorage",
    "value": "xxx",
    "slotSetting": false
  }

Additionally, I've also disabled the scale count of my function app, to prevent Azure from scaling out instances.

Despite all these configurations, my function app continues to dequeue and process multiple messages concurrently.

Looks like the application not reading from the host.json file on azure environment.

Has anyone encountered this ?

Answer 1

The host.json file is located under the "App Files" section of the Azure Function App in the Azure portal.

Despite having the correct queue configurations, the application initially fails to dequeue messages sequentially. However, if I place the same set of messages in the queue over 4 or 5 times, the queue trigger works correctly. In this case, the function processes one message at a time, only dequeuing the next message after completing the current one. This behavior is consistent each time.