Details about Scale-in behavior with target-based scaling in Azure Functions Premium Plan

Question

Context:

When using Azure Functions Premium plan, scaling is done via "event-driven scaling"

When using a Storage Queue Trigger, "target-based scaling" is used by default.

In its documentation Target-based scaling, Microsoft describes that the equation used to determine the desired instances for scaling is:

Desired instances = Event source length / Target executions per instance

When using Storage Queue Trigger, the Event source length is the amount of messages in the Storage Queue and Target executions per instance is the extensions.queues.batchSize property defined in host.json.

My understanding:

So let's say, I want to configure scaling in a way that each intstance only processes one queue message at a time.

In this case, i would set the extensions.queues.batchSize to 1.

So now, Azure Functions premium plan would scale-out an instance for each queue message (until the maximum amount of instances is reached, from then on it will wait for processing messages to complete).

As stated above, the target-based scaling uses the messages in the queue to determine the needed amount of instances.

But after they start processing, they are no longer in the queue and so the target-based scaling would immediately vote to scale-in the scaled-out instances, wouldn't it?

Example:

• Azure Functions Premium plan is configured to have 1 instance "always ready" and scale up 4 additional instances
• There are 3 messages in the queue the queue trigger reads from
• The scaling is configured as described above (1 message per instance at a time)
• So now, the target-based scaling will scale-out two additional instances so that all three messages can be processed concurrently
• But now, the queue is empty. So the target-based scaling will now vote to scale-in back to the initial 1 instance

Questions:

1. Is my understanding of the scale-in behavior correct or do I miss something?
2. If it is correct, do the instances marked for scale-in actually complete their work or will they be shut down after some time? I have read about drain mode and graceful shutdown, but I am not sure if I understand it correctly.
3. If it does not actually let the instances complete their work (shut down after some time even if it did not complete), then how to ensure they are completed? Writing a queue message again during shut down would result in a queue item, which would result in a scale-out again. So it just goes back and forth?

About my function:

My function is used to process some word files (docx). Usually, most jobs complete within seconds or minutes. However, there can be jobs that run a few hours.

I have read about Durable Functions as well, but I am not sure if it solves my problem since the target-based algorithm would be the same for Durable Functions and "Normal" Functions.

Answer 1

Hi Dean,

Check below steps to understand scale-in behavior with target-based scaling:

When the queue is empty, the target-based scaling mechanism will determine that instances that are needed and will scale down the number of instances accordingly, but also ensures the ongoing work is completed through Drain Mode.

Azure Functions on the Premium Plan have Drain mode enabled by default which means when scaling-in occurs, the instances that are being shut down will have time to complete all the active processes.

If a function is in progress when the scale-in happens, Azure Functions will not terminate the instance immediately. Instead, the function will be given a grace period to complete the in-progress requests.

Graceful shutdown: When the function scales down the instances, it waits for the function to complete its current execution before terminating the instance.

Drain mode: When an instance is being scaled down, it will still handle any in-progress requests but will not take on new tasks.

When the host is in drain mode:

1. It stops listening for new incoming requests,
2. Cancellation token is passed as a parameter to the function invocation,
3. A scale-in operation will be performed.

Durable Functions can indeed help with long-running tasks, but it uses the same Azure Functions scaling model. If you have multiple messages in a queue, Azure Functions will still try to scale out to process the messages concurrently.

Hope this helps.

If the answer is helpful, please click Accept Answer and kindly upvote it. If you have any further questions about this answer, please click Comment.