Understanding ‘nested’ management pack discoveries and how they impact total discovery time?
Most management packs require discoveries be run to figure out which systems are hosting the objects we wish to monitor with the management pack. Only after the discoveries find our target objects do processes kick in to deliver appropriate monitors/rules so that monitoring begins.
In some cases, discoveries are ‘nested’. What does ‘nested’ mean? Put simply, discoveries that are ‘nested’ are in relation to one another. Generally a broader discovery runs that finds systems hosting the general class of object to be monitored. Based on the results, more specific discoveries run to determine details about the object found by the first discovery run. Before monitoring is fully in place, all of these ‘nested’ discoveries have to run and return data for a system hosting the monitoring object. This approach may take longer to get through the entire discovery process – but the benefit is that rules/monitors are only delivered to systems that specifically have the components of interest.
Confusing? Not really – an example will clear things up. Let’s take a look at the IIS 2003 Management Pack as an example. To understand the discoveries and how they related to one another the first thing we need to do is find the ‘root’ discovery. There are a couple of approaches to this – using the OpsMgr UI, reading through the XML or using MP Viewer.
I don’t like using the OpsMgr UI for this for a couple of reasons. First, some object discoveries are ‘hidden’ from the list requiring you to go searching for them. Kevin has a good blog about doing this – click here. Second, the UI will sometimes show the same discovery rule multiple times – once for each type being discovered as shown.
This can be a bit confusing. Just remember, even though the rule is listed multiple times, these are all the same discovery rule.
You can also review the management pack in raw XML but this requires advanced knowledge and isn’t a good option for many administrators.
That leaves us with MP Viewer (make sure you use the latest version – 1.7 as of this blog entry). Opening a management pack in MP viewer will show us any discoveries in that management pack – one line per discovery!
The first thing we need to do is find out which of our discoveries is the ‘root’ discovery. As mentioned ‘root’ discoveries will have the broadest target. For the IIS 2003 Management Pack, there are 18 discoveries. From the list we find only one that targets the Windows Server class – since this is the broadest class targeted, this is our ‘root’ discovery.
Notice the list of discovery targets though – most discoveries in the list target the ‘IIS 2003 Web Server’ class. So the question at this point is – which discovery creates and populates the ‘IIS 2003 Web Server’ class. Glad you asked, lets keep going.
Now that we have the ‘root’ discovery identified the next question is – what class does this ‘root’ discovery create and populate? Looking at the XML tab for this discovery we can easily see that this discovery creates the InternetInformationServices.2003.ServerRole class.
Very quickly we know that our ‘root’ discovery doesn’t create our ‘IIS 2003 Web Server’ class – but it does create the IIS 2003 Server Role class. So far in the discovery ‘cascade’ this is the only class created. So is there a discovery that targets the IIS Server Role class? There are three.
The License Discovery rule doesn’t sound of interest to us at this point, the IIS All Components rule is disabled by default, so that’s not it. that leaves us with the IIS Base Class Discovery. What does the discovery for this class do and what classes are created and populated as a result? Taking a look at the XML shows us that this one discovery results in several classes being populated – IIS FTP, IIS NNTP, IIS SMTP Virtual Server and several others – along with them is our class of interest – IIS 2003 Web Server (shown in Red)
- <Discovery ID="Microsoft.Windows.InternetInformationServices.2003.DiscoverBase" Enabled="onEssentialMonitoring" Target="Microsoft.Windows.InternetInformationServices.2003.ServerRole" ConfirmDelivery="false" Remotable="true" Priority="Normal">
<Category>Discovery</Category>
- <DiscoveryTypes>
- <DiscoveryClass TypeID="Microsoft.Windows.InternetInformationServices.2003.ServerRole">
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.ServerRole" PropertyID="MinorVersion" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.ServerRole" PropertyID="MajorVersion" />
</DiscoveryClass>
<DiscoveryClass TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.FTPServer" />
- <DiscoveryClass TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.FTPSite">
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.FTPSite" PropertyID="SiteID" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.FTPSite" PropertyID="Description" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.FTPSite" PropertyID="ServerBindings" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.FTPSite" PropertyID="SecureBindings" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.FTPSite" PropertyID="AllowAnonymous" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.FTPSite" PropertyID="AnonymousOnly" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.FTPSite" PropertyID="AnonymousUsername" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.FTPSite" PropertyID="LogFileDirectory" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.FTPSite" PropertyID="Path" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.FTPSite" PropertyID="ConnectionTimeout" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.FTPSite" PropertyID="MaxConnections" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.FTPSite" PropertyID="LogType" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.FTPSite" PropertyID="URL" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.FTPSite" PropertyID="LogFileType" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.FTPSite" PropertyID="LogFilePattern" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.FTPSite" PropertyID="LoggingEnabled" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.FTPSite" PropertyID="PerfCounterInstance" />
</DiscoveryClass>
<DiscoveryClass TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.WebServer" />
<DiscoveryClass TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.SMTPServer" />
- <DiscoveryClass TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.SMTPVirtualServer">
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.SMTPVirtualServer" PropertyID="SiteID" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.SMTPVirtualServer" PropertyID="Description" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.SMTPVirtualServer" PropertyID="ServerBindings" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.SMTPVirtualServer" PropertyID="SecureBindings" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.SMTPVirtualServer" PropertyID="DefaultDomain" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.SMTPVirtualServer" PropertyID="BadMailDirectory" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.SMTPVirtualServer" PropertyID="PickupDirectory" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.SMTPVirtualServer" PropertyID="QueueDirectory" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.SMTPVirtualServer" PropertyID="LogFileDirectory" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.SMTPVirtualServer" PropertyID="EnableReverseDNSLookup" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.SMTPVirtualServer" PropertyID="ConnectionTimeout" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.SMTPVirtualServer" PropertyID="MaxConnections" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.SMTPVirtualServer" PropertyID="MaxMessageSize" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.SMTPVirtualServer" PropertyID="NTAuthenticationProviders" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.SMTPVirtualServer" PropertyID="RemoteSMTPPort" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.SMTPVirtualServer" PropertyID="RemoteSMTPSecurePort" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.SMTPVirtualServer" PropertyID="RemoteTimeout" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.SMTPVirtualServer" PropertyID="LogFileType" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.SMTPVirtualServer" PropertyID="LogFilePattern" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.SMTPVirtualServer" PropertyID="LoggingEnabled" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.SMTPVirtualServer" PropertyID="PerfCounterInstance" />
</DiscoveryClass>
<DiscoveryClass TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.NNTPServer" />
- <DiscoveryClass TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.NNTPVirtualServer">
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.NNTPVirtualServer" PropertyID="SiteID" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.NNTPVirtualServer" PropertyID="Description" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.NNTPVirtualServer" PropertyID="ServerBindings" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.NNTPVirtualServer" PropertyID="SecureBindings" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.NNTPVirtualServer" PropertyID="Path" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.NNTPVirtualServer" PropertyID="NewsDropDirectory" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.NNTPVirtualServer" PropertyID="NewsFailedPickupDirectory" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.NNTPVirtualServer" PropertyID="NewsPickupDirectory" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.NNTPVirtualServer" PropertyID="LogFileDirectory" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.NNTPVirtualServer" PropertyID="AllowAnonymous" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.NNTPVirtualServer" PropertyID="ArticleTable" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.NNTPVirtualServer" PropertyID="MaxMessageSize" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.NNTPVirtualServer" PropertyID="NTAuthenticationProviders" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.NNTPVirtualServer" PropertyID="ClientPostHardLimit" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.NNTPVirtualServer" PropertyID="ClientPostSoftLimit" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.NNTPVirtualServer" PropertyID="FeedPostHardLimit" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.NNTPVirtualServer" PropertyID="FeedPostSoftLimit" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.NNTPVirtualServer" PropertyID="LogFileType" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.NNTPVirtualServer" PropertyID="LogFilePattern" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.NNTPVirtualServer" PropertyID="LoggingEnabled" />
<Property TypeID="IISCommon!Microsoft.Windows.InternetInformationServices.NNTPVirtualServer" PropertyID="PerfCounterInstance" />
</DiscoveryClass>
</DiscoveryTypes>
- <DataSource ID="DS" TypeID="Microsoft.Windows.Server.IIS.IISDiscoveryDataSource.2003">
<PeriodInSeconds>3600</PeriodInSeconds>
<DiscoverySourceType>0</DiscoverySourceType>
<DiscoverySourceObjectId>$MPElement$</DiscoverySourceObjectId>
<DiscoverySourceManagedEntityId>$Target/Id$</DiscoverySourceManagedEntityId>
<ComputerPrincipalName>$Target/Host/Property[Type="Windows!Microsoft.Windows.Computer"]/PrincipalName$</ComputerPrincipalName>
<DiscoveryType>1</DiscoveryType>
<LowerDiscoveryPercentage>0</LowerDiscoveryPercentage>
<UpperDiscoveryPercentage>100</UpperDiscoveryPercentage>
</DataSource>
</Discovery>
Notice also that the properties that are discovered for each class are shown here as well.
So now we have drilled through the discoveries. From our review we have learned:
1. ‘Root’ discovery – IIS Role Discovery Rule – creates and populates
IIS 2003 Server Role class
2. IIS 2003 Server Role class – creates and populates several classes,
including IIS 2003 Web Server Role
3. 12 discoveries run against the members of the IIS 2003 Web
Server Role class.
A logical question out of this – we saw that a number of classes were created and populated in step 2 above – but we don’t see any discoveries targeted against those classes. Remember, not all classes that are discovered have a discovery targeted to them. Some classes are created for the purpose of targeting rules and monitors. Some classes are created for the purpose of targeting further discoveries. Some classes are created for both purposes.
OK, so now we understand how the discoveries relate to each other. But now we need to understand how long it will take for each of these discoveries to create and populate their classes and get the job of monitoring IIS started for the environment. The answer here is easy. In MP Viewer we notice that beside each discovery we show the frequency at which the discovery runs – 3600 minutes for each discovery show – or every hour.
Knowing the timings we can figure out how long, at longest, it will take for all objects to become discovered in the environment. When the MP is first imported, the ‘root’ discovery runs right away. After it runs, the resulting class will be populated and the next level of discovery will be deployed. So on we go until all discoveries are deployed.
Going by the frequency for each discovery, the maximum time for all discoveries to run and for an object to be fully discovered/monitored would be 3 hours – this is true because there are three layers of discovery, each running on an hour interval. You also need to factor in some time to accommodate for internal works of OpsMgr as classes are created, config changes are processed, new management packs are requested by agents, etc.
A couple of more things to factor in here – when the MP is first imported the ‘root’ discovery runs right away. Once imported, for any new agents that are added, the ‘root’ discovery will be on a schedule and likely will not run right away. Bottom line? The discovery cascade will run more quickly when the MP is first imported that for subsequent runs. Further, the discoveries shown here are running every hour. This is generally far too frequent for environments that have any size at all. Depending on the environment I generally recommend moving discoveries to no more than every 12-24 hours. There are a lot of factors that go into whether or not to modify a discovery and, as such, I don’t suggest modifying discovery schedules without a good reason. So what are some good reasons? Console sluggishness can be caused by too frequent discovery and too much churn in the discovery data. Environments monitoring lots of a particular type of object (in this case IIS) may see problems due to the flood of data that comes in with the discoveries. I discuss one example where it would be beneficial to modify discovery frequency in my recent blog post here If you think you may be adversely affected by too frequent discoveries, it would be worth a call to PSS before carte blanche changing discovery frequency.
One last thing as I wrap this up. In some environments, certain SLA’s are in place to ensure monitoring begins within a certain time frame of agent install. Because of this you might be a bit hesitant to modify discovery frequencies even if needed. It’s worth noting a couple of things.
1. Monitoring will begin in some form after the first discovery in the
cascade completes – so the agent will be monitored fairly quickly.
Additional monitoring will be deployed as the discovery cascade
continues to fill out.
2. If it is vital to ‘get through the discovery cascade’ quickly but you
also find you need to adjust the discovery frequency for your
environment, you can still make this work and meet SLA’s – getting
through the ‘cascade’ can be expedited by simply restarting the
health service on the target agent. At each health service restart
the agent will process through all assigned discoveries. So, if
needed, you can use this method of restarting the health service a
few times and watching for appropriate events in the opsmgr
event log to expedite the agents way through the discovery cascade.
Comments
Anonymous
June 09, 2009
The RMS is responsible for maintaining a master list of agent configurations for the management group. Anonymous
June 11, 2009
Steve I don't follow you when you say that newly added agents are discovered slower. In my experience when a new agent is installed and the built in discoveries populates the base classes (i.e HealthService and so on) agents start receiving new MPs based on discovered objects and disocveries tartget. As soon as the the new MP is loaded the discoveries are run and so on. What you describe, once again in my experience, is true when you add a new app to an already managed server (say you install IIS or add a SQL Instance), it is also true with multihoming for the reasons you explained and (probably) for the cookdown mechanics that rationalizes disocoveries as well as other montioring rules.Anonymous
June 11, 2009
There is a progression here. When a new MP is first loaded into an environment with existing agents, no discoveries need to run other than the ones from the new MP - so it is fastest. If we are installing new agents, we have to start at the beginning and run base discoveries followed by all MP discoveries - a bit slower.