Resource Management in Concurrency Runtime – Part 3

In my previous blog post, I talked about the dynamic migration concept of the Concurrency Runtime’s (ConcRT) Resource Manager (RM). Today I will be demonstrating that concept in action and will focus on its performance characteristics.

A Demonstration of Dynamic Migration

The scenario that we are going to use to demonstrate the resource manager’s dynamic core migration concept involves two schedulers running different kinds of workloads in phases. The schedulers will be created with default policy on an 8 core machine (MinConcurrency=1, MaxConcurrency=8). The workload of schedulers in each phase will be characterized by one of the following:

i) Both schedulers have an equal amount of work

ii) One scheduler has work, the other one has no work

iii) One scheduler has more work than the other scheduler

Here are the phases and the recorded performance of the Resource Manager in detail:

Phase1: In this phase both schedulers have equal amount of work. No dynamic migration happens and schedulers keep their proportionally allocated resources. Note that both schedulers have 4 resources.

Phase2: Only Scheduler1 has work here. This phase starts at 2.246s; RM reacts at 2.309s by allocating all resources to Scheduler1. Note that Scheduler1 has 8 resources of which one is shared with Scheduler2. That is because resource manager cannot take away cores below the MinConcurrency value and has to have at least 1 core assigned to Scheduler1. However, Scheduler1 has no work and the resource is idle therefore Scheduler2 makes use of it without interruption.

Phase3: This is similar to Phase2, this time only Scheduler2 has work. The phase starts at 4.508s; RM reacts at 4.602s by allocating all resources to Scheduler2.

Phase4: Here both schedulers have work but Scheduler1’s work is more than Scheduler2’s. The phase starts at 6.833s; RM first balances resource usage among schedulers at 6.911s taking into account the workload of each. This continues until Scheduler2 completes all of its work. At 8.112s all resources are allocated back to the Scheduler1.

Focusing on Dynamic Migration Reaction Time

In order to give more details on the dynamic migration reaction time, a zoomed in view of Phase4 will be examined.

Here the first core migration happens after 78ms where RM balances the resources between the two schedulers. This reaction time depends on a number of factors:

1) The statistics polling interval in resource manager

All core migration decisions are done after resource manager polls the schedulers. With the current implementation poll interval is 100ms, therefore the reaction time can be anywhere between 0 and 100ms.

 

2) The statistics from the schedulers

Statistics is the representation of scheduler workload in resource manager including task completion rate, incoming task rate and total task queue size. The faster the growth in statistics the faster the scheduler will get extra resources. This also implies that with a slower growth resource manager may delay allocation to the upcoming polls.

 

3) The availability of resources

Resource manager will only migrate if there is an available resource. Resources are considered available if other schedulers have declared their resources idle or have declining statistics. In other words, resource manager’s reaction will be delayed until there is an available resource.

 

After rebalancing, the statistics don’t change until 8.112s therefore resource manager continues with the current allocation. Whenever Scheduler2 completes all work, all resources are allocated back to Scheduler1.

Next Steps and Feedback

I would love to hear feedback regarding the blog content or areas of interest with respect to Concurrency Runtime to have a guided focus on blogging.