Collection Evaluation Overview - Configuration Manager

Collection evaluation is the method by which Configuration Manager (ConfigMgr) takes the collection rules you have defined and updates collection membership based on these rules. Collection evaluation happens differently and at different times based on the administrator-defined configuration of a site and collection.

Some advice for the management of collections can appear counter-intuitive. For example, the general recommendation is to limit the number of collections that update frequently to avoid creating performance problems. However, creating frequently updating collections can be very convenient, especially in light of most ConfigMgr functionality being dependent on collections. The performance impacts and business requirements need to be carefully considered to come up with an appropriate collection evaluation plan.

This blog post seeks to explain how collection evaluation works in ConfigMgr 2012 (and ConfigMgr current branch) including the new concept of a collection evaluation graph.

At a high level, a collection will be evaluated based on the flow identified in Figure 1.

[caption id="attachment_355" align="aligncenter" width="500"] Figure 1 - High-Level Collection Update Process[/caption]

Hierarchy Considerations

A ConfigMgr environment may have one or more sites based on requirements.  There are three types of sites that can be created:

• Central Administration Site (CAS) which sits at the root level of the ConfigMgr environment. This site does not manage clients but does manage lower level sites.
• Primary Sites sit immediately below the CAS and receive instructions replicated from the CAS database on which they act. Primary Sites manage clients and are the only site that evaluates collections.
• Secondary Sites sit below a primary site and act as a proxy for that primary site. Data on the secondary site is exclusively populated using replication from the associated primary site.  Secondary sites do not evaluate collections.

Given that a CAS does not evaluate collection membership, it needs to send a request to each primary and request a collection update. The primary sites evaluation the collection update and send the results back to the CAS.

Figure 2 demonstrates an administrators experience when requesting a manual collection update.

[caption id="attachment_365" align="aligncenter" width="499"] Figure 2 - Administrator Experience During a Manual Collection Update from a CAS[/caption]

Collection Evaluation Types

Collection evaluation is handled by different threads depending on the type of evaluation. These threads are:

• Primary – when a collection update is scheduled;
• Auxiliary – when a manual collection update is triggered for a collection with dependent collections;
• Single – when a manual collection update is triggered for a collection with no dependent collections;
• Express – processes incremental collection evaluation.

Table 1 describes the collection evaluation triggers:

Trigger	Description
Manual	This is the highest priority collection evaluation.  If an administrator chooses to perform a manual collection evaluation, then the next evaluation thread that becomes available will be assigned to a manual evaluation.
Scheduled	The process of scheduled evaluation is identical to that of manual evaluation with the exception of course that the evaluation is now not event-driven but time-driven.
Incremental	Incremental Collections are a collection evaluation schedule that is similar to the Delta collection updates that were possible in ConfigMgr 2007.In ConfigMgr 2007, delta updates only added resources to a collection when the device was initially discovered. So if a collection query was based off information that would only be updated later (i.e. hardware inventory), then the resource would only be added to or removed from a collection during a scheduled collection update.In ConfigMgr 2012, incremental collections work as administrators expected in ConfigMgr 2007. Any update to a resources objects are monitored and updated in collections configured for incremental updates.Incremental evaluation uses a collection evaluation graph to evaluate and update dependent collections if an update to the incremental collection membership changes.
Staging	Since ConfigMgr 2012, all the collections should be dependent on All Systems or All Users and Groups directly or indirectly. Both of these collections perform a full collection evaluation at 4:00 AM every day. As per the full evaluation cycle, if a change occurs to either of these collections it will trigger updates of dependent collections based on a Full Collection Graph (explained later).

Table 1 – Collection Evaluation Types

Evaluation Process

As each individual collection is evaluated, the following steps are performed:

1. Execute each collection query
2. Add any systems which are direct members
3. Evaluate all Include collections.
4. A logical AND is performed between this list and the limiting collection.
5. The exclude collections are then evaluated.
6. The results of this step are then logically OR’d against the result set above.
7. The changes are then either written to, updated or deleted from the database.
8. Trigger any dependant collections to update as well (collections that refer to this collection using an include rule, and exclude rule or collections that are limited to this collection)

Collection Evaluation Graph

The collection evaluation graph is a new concept since ConfigMgr 2012. Previously in ConfigMgr 2007 a collection evaluation was an isolated event resulting in updates only to the targeted collection. In ConfigMgr 2012 a collection evaluation involves the targeted collection and any related collections.

The collection evaluation graph is built differently based on the type of collection evaluation being performed. It is important to understand the different ways in which collections will be evaluated so that appropriate collection design decisions can be made. In general, there are 2 types of evaluation graphs that are built by ConfigMgr;

• Incremental, or;
• Full.

When collection evaluation starts ConfigMgr will build a graph which will include all collections which could possibly need evaluating as a result of changes to the relevant collection. The collection evaluator then builds all of these into the Collection Evaluation Graph starting from the highest level within this cycle.  It will then start to move through the graph in order evaluating each collection membership in turn.  When the collection has been evaluated then any collections below which would NOT be affected by this collection evaluation cycle are removed from the evaluation graph.

If one or more of the collections being evaluated has an Include or an Exclude rule then the collection being included or excluded will also have been added to the graph together with any collections limited by that collection.  During the evaluation of the include and exclude collections should there be any changes then the graph will continue on that branch before returning to the main branch.

During an incremental collection evaluation, the collection evaluation graph will only map referencing collections if they are enabled for incremental evaluation. If an incremental collection is limited to a collection that is not enabled for incremental evaluation, it will be evaluated based on the existing membership of the limiting collection. For example, in Figure 3 even though there are newly discovered resources applicable to all collections, only All Servers and All Servers with Windows Server 2008 R2 are updated while the other collections are not evaluated because the All Servers with Windows Server 2012 R2 collection is not enabled for incremental evaluation.

[caption id="attachment_366" align="aligncenter" width="580"] Figure 3 - Incremental Collection Evaluation Graph Example[/caption]

When a collection evaluation is triggered manually or based on a schedule, a full collection evaluation graph is built of all dependent collections. All collections that reference the collection that is updating or subsequent collections are included. ConfigMgr will continue to evaluate down the graph as long as updates to the collections being processed occur. Figure 4 demonstrates how the collection evaluation graph includes all applicable collections when a scheduled or manual collection update request is made for the All Servers collection. In the example, 2 new DNS servers running Windows Server 2008 R2 and Windows Server 2012 R2 have been discovered since the last collection evaluation. The two new resources are in scope of the membership queries of all collections shown resulting in all collections updating.

[caption id="attachment_367" align="aligncenter" width="580"] Figure 4 - Full Collection Evaluation Graph Example[/caption]

Conversely, Figure 5 demonstrates that not all collections are always evaluated during a full evaluation. The collection evaluation graph will only continue to evaluate dependent collections if an update occurs to the corresponding referenced collection. In the example, the installation of DNS on the Windows Server 2012 R2 server would make it a member of the All Windows Server 2012 R2 Servers with DNS Server collection, however because no update occurred to its limiting collection the collection is not evaluated during this evaluation. It would be evaluated during the next incremental evaluation cycle (because it is an incremental collection).

[caption id="attachment_376" align="aligncenter" width="580"] Figure 5 - Full Collection Evaluation Graph Example 2[/caption]

For some more technical information about how collection evaluation works, see the recording of the ConfigMgrDogs Troubleshoot ConfigMgr 2012 session at TechEd Australia.

I'd also like to make a shout out to Jason Wallace who originally put some of this information together.

Comments

• Anonymous
  September 14, 2017
  Interesting post, the part "Any update to a resources objects are monitored and updated in collections configured for incremental updates." isn't fully accurate though. Only changes that have explicitly been added to this mechanism are taken into account.Most notable exception is the absence of updates on CI state. You can validate that by creating a CI, making sure it switches state, and validate the incremental updates on a collection that is based on CI state. You'll notice that the collection update is none-the-wiser of that state change.