MIIS 2003 Capacity Planning
Applies To: Windows Server 2003 with SP1
Download Instructions
This document is available for you to download from the Web at https://go.microsoft.com/fwlink/?LinkId=55412.
Overview
This document presents capacity planning recommendations for the configuration of a server hosting Microsoft Identity Integration Server (MIIS) 2003. The recommendations presented here are based on tests designed to determine the most effective use of central processing units (CPUs), memory capacity, disk access, and disk capacity on a server hosting MIIS 2003. Other factors, such as database performance and the impact of the network, are also included.
MIIS 2003 gathers identity-related data from a collection of data sources and then synchronizes and stores the data in its own database before exporting the synchronized data back to data sources. During this process, many potential factors can impact the performance of the MIIS 2003 environment. For example, the server running MIIS 2003 uses a SQL server database. Each time the server running MIIS 2003 needs to read from or write information to the database, the performance of the SQL server directly impacts the performance of the server running MIIS 2003. If the SQL server is located remotely, then each of these transactions must occur through a network connection. If the network segment used is operating near maximum capacity, then available bandwidth may be insufficient, further limiting the overall performance of the MIIS 2003 infrastructure.
This document presents the results of the tests and makes specific recommendations regarding hardware configuration to maximize the performance of the MIIS 2003 server. An overview of each test and a general discussion of the results are included so you can see exactly what was tested and have your own discussions about the data presented. Factors attributed to the performance of the external data sources that are being synchronized are outside the scope of this discussion.
Executive Summary
A group of tests were performed to determine how the configuration of the hardware that hosts the MIIS 2003 environment affects the performance of MIIS 2003. Five different aspects of configuration were tested: processor configuration, database location, memory, disk drive configuration, and network bandwidth.
Different processor configurations were tested to determine whether the number of processors or the speed of the processors most significantly affected the performance of the MIIS 2003 server. Database tests were performed to determine how the location of the MIIS 2003 database, whether on the same server as MIIS 2003 or on a remote SQL server, impacted the performance of MIIS 2003. Memory tests were performed to determine how MIIS 2003 utilizes memory so recommendations could be made regarding how to determine the amount of RAM necessary to optimize performance of the MIIS 2003 server. Different disk configurations were tested to determine how the number of spindles and the location of log and database files affected performance. During tests that utilized the network, network traffic was monitored to measure the impact of various types of operations on network bandwidth.
The results of the tests indicated that an optimal server configuration for MIIS 2003 should follow these guidelines:
Processors: Use a dual-processor server that contains the fastest processors you can acquire. The speed of the processors has more impact on the performance of MIIS 2003 than the number of processors. No significant performance gains were seen when switching from a dual-processor platform to a quad-processor platform where the other configuration aspects remained constant. Significant performance improvements were seen when a 3.2-GHz dual-processor platform was compared to a 2.8-GHz quad-processor platform.
Database Location: If your administrative and security policies will permit it, locate the SQL server hosting the MIIS 2003 database on the same server that is hosting MIIS 2003. Even if the SQL server is optimized for peak performance, if it is located remotely, any performance gain realized by the optimization will not overcome the latency introduced by the network when transmitting the data between the MIIS 2003 server and a remote SQL server. You might have other considerations beyond performance, such as hosting SQL Server in a clustered environment, that might override this recommendation.
Memory: At a minimum, servers should be configured with at least 1 GB of RAM for each processor. If you will be managing 50,000 or more objects from more than two data sources, then it is recommended that you use 2 GB of RAM for each processor.
Disk Configuration: Use a RAID array with as many spindles as possible on the server that hosts the MIIS 2003 database. Locate the database and log files on separate volumes.
Network: The impact on available bandwidth is negligible as long as you have LAN connectivity (10 megabits per second or greater) between the MIIS 2003 server and the SQL server that hosts the MIIS 2003 database, even when synchronizing large numbers of objects (100,000 were tested). If you use a remote SQL server to host your MIIS 2003 database, it is recommended that the speed of the connection between the two servers is greater than 5 Mbps.
These recommendations are based on the results of the tests presented in this document. Other external factors, such as budget constraints, administrative, or security policies within your organization, might dictate that other configuration options be used. If other factors apply to your environment, use the recommendations presented in this document as a starting point for the planning process and to provide examples about how to approach making some of these configuration decisions.
Capacity Testing Summary
The tests presented here focus on configuration options for the server hosting MIIS 2003 and help answer common questions with regards to these options:
Processor configuration: Which is more beneficial: To invest in servers with a large number of processors or invest in servers that have fewer, but faster processors?
Database performance: How does the location of SQL server that hosts the MIIS database affect performance? Are there advantages or disadvantages associated with using SQL server in a remote location?
Memory utilization: How do you calculate the amount of memory the server will need? Is it related to the number of objects being managed? The number of data sources being synchronized?
Disk drive performance: What disk configuration provides the best performance?
Network bandwidth utilization: How is performance impacted by network traffic? How do MIIS 2003 operations affect the available bandwidth on my network?
Processor Configuration
The processor performance of the MIIS 2003 server significantly affects the performance of MIIS 2003. During import operations, each object must be examined to determine whether or not any connector filter synchronization rules need to be applied. During synchronization operations, join and projection rules are processed for each object in addition to any rules extension code. By themselves, most of these operations are generally quite simple. Considered individually, they do not amount to a significant use of processor time. However, when you consider the fact that some staging and synchronization operations in some deployments involve hundreds of thousands of objects, these operations can quickly amount to a significant load on the server.
In order to gauge how these operations impact processor utilization, two servers were configured to host MIIS 2003. The first server was configured with dual-processors operating at 3.2 GHz, and the second server was configured with four processors running at 2.8 GHz. For some tests, two of the four processors running at 2.8 GHz were shut off so a direct comparison could be made between two and four processors running at the same clock speed.
A number of steps were taken to minimize any outside influences on the tests being performed. For all tests, the servers used the same disk array for storage in order to avoid skewing the tests based on differences in disk access time. Similarly, an isolated network was used to connect the various servers involved in the test. The only traffic on the network was that being generated by the tests. This helped avoid skewing the tests based on network latency created by traffic from other systems on the network. A pool of 500,000 user objects was created for use in the processor tests. Each user object had 25 attributes defined. Each time a test was run, a group of users from this pool was used. This helped ensure that the test results would not be skewed due to inconsistent test data. Each test used the same pool of users and maintained the same ratio of user objects-to-attributes.
For the first test, 10,000 objects were first staged and then synchronized. Performance statistics on the total time it took to complete the operations and the number of operations per second were recorded. The next test used 50,000 user objects, the test after that used 100,000, and so forth until 500,000 objects were tested. All of the performance data was collected and analyzed to determine how different numbers of objects impacted the utilization of the processor.
SQL server performance
One option of any MIIS 2003 installation is whether to install Microsoft SQL Server 2000 on the same server that is hosting MIIS 2003 or to use a remote SQL server database and access it over the network. Some customers install MIIS 2003 and Microsoft SQL Server 2000 on separate servers to balance the workload of MIIS 2003 operations across multiple servers.
Other factors can dictate the use of separate servers. Some organizations dedicate a SQL Server team to oversee the operation of all databases within the organization. For management and disaster recovery purposes, the SQL Server team dictates where the database exists. Also, some organizations have clustered SQL database servers that are used to consolidate corporate data stores. In both cases, databases are located according to internal policy rather than performance considerations.
An alternative opinion on the use of remote SQL servers is that network latency during communication between the MIIS 2003 server and the SQL server will have a greater impact on the overall performance than the added workload of hosting both MIIS 2003 and SQL on the same server. Thus, it is more efficient to install both services on the same server if the administrative policies of your organization permit it.
To test these scenarios, MIIS 2003 and SQL Server were installed on one quad-processor server and one dual-processor server. Two management agents were configured on each MIIS 2003 server. The data sources were two text files that each contained 100,000 user objects with 15 attributes defined. During the tests, one management agent was used to stage and project the users into the metaverse. The other management agent was used to stage and then join the metaverse objects. These tests were performed on different combinations of server configurations (such as when MIIS 2003 was installed on the dual-processor server while SQL Server was installed on the quad-processor server or when both MIIS 2003 and SQL Server were installed on the quad-processor server). Performance data was collected for each test and analyzed to determine which configuration provided the best performance.
Disk Configuration
The configuration of the disk subsystem has a significant impact on the performance of MIIS 2003. MIIS 2003 uses a SQL server database as it central data store. Most MIIS 2003 operations require access to the database. By hosting the SQL database on a server that is configured with an optimized disk subsystem, you can improve performance in your MIIS 2003 environment.
To test this, two server platforms were used to stage and then synchronize user objects (in various increments, from 10,000 to 500,000). The disk drive configuration of the first platform consisted of 2 internal hard disks that hosted both MIIS 2003 and the SQL server database used by it. The second platform was configured to use an external, SCSI attached RAID 0+1 containing ten spindles. For this configuration, the database and log files were stored on separate volumes to help optimize performance. Operations per second were monitored to determine the difference in performance.
Memory Utilization
As more identities are managed, the MIIS 2003 server uses more memory. Each identity is stored as an object and the amount of memory required depends on the number of objects being managed and the number and type of attributes associated with each object.
To test the relationship between the number of objects managed and the amount of memory used, ten management agents were created. Each management agent was used to process batches of objects, ranging from 10,000 to 500,000 objects. For each test pass, the first management agent staged and projected the objects into the metaverse. The remaining nine management agents were used to join those metaverse objects. Memory usage was then monitored to see how the number of objects being processed related to the amount of memory used.
The test results indicated that servers should be configured with at least 1 GB RAM for each processor. If more than 50,000 objects are being managed and more than two data sources are being used, then it is recommended that you use 2 GB RAM for each processor.
Database Size
During the processing of various tests for this document, database size was monitored to determine how the different operations affected the size of the MIIS 2003 database. Although this document does not provide any specific guidelines around managing database size, the section "MIIS 2003 Capacity Planning Test Summary - Database Size" contains a discussion of the database growth data recorded during the testing of other resources.
Trends can be found in this data that may be used as a basis for estimating how various operations might affect the size of the MIIS 2003 database. The discussion around these trends is not meant to provide definitive data on the relationship between database size and the operations tested. The intention is to provide an example of how you might go about examining test data based on your own environment and make estimates based on that information.
Network Utilization
Due to the distributed nature of an MIIS 2003 environment, it is likely that most of the different components will be connected by your organization's network. During import and export operations, data needs to flow between the MIIS 2003 server and one or more connected data sources. If the MIIS 2003 database is on a remote SQL server, then data also must flow between the MIIS 2003 server and the SQL server. Because virtually every type of MIIS 2003 operation requires access to the database, network traffic is generated if the database is remote.
MIIS 2003 deployments can potentially require managing millions of objects. All of this data must flow between the data sources that host those objects, the MIIS 2003 server, and the MIIS 2003 database.
With such potentially large sets of data and the possibility of connecting many data sources, some administrators are concerned about how much network bandwidth is consumed during import, export, and synchronization. This is a notable concern for administrators of systems that do not support delta import. Delta-based operations are more efficient because MIIS 2003 processes only objects that have changed. For data sources that do not support delta-based operations, every object must be processed during import operations, including objects that are unchanged.
During a number of the tests performed for the research presented in this guide, network traffic was monitored to observe the impact of operations such as the synchronization of 100,000 objects when MIIS 2003 is using a remote SQL Server to host its database. Other tests were monitored to observe the impact of importing data from a remotely located data source.
The results of these tests demonstrated that network impact is quite low. Experiences in the test environment indicated that as long as you have LAN connectivity (10 Mbps or more) the overall impact on bandwidth is negligible. Reliability of the connection proved to be more important than speed.
Conclusions and recommendations
Based on the results of the tests described in this document, the following configuration options are recommended:
Server Configuration
Use a dual-processor platform with the fastest processors that your budget allows. The tests demonstrated that a dual-processor platform will outperform platforms with additional processors.
Note
If your environment frequently requires a large number of concurrent staging or export operations, a server with more than two processors might provide better performance. If possible, test this configuration before making a decision.
Database Configuration
If the administrative policies of your organization permit it, install MIIS 2003 and SQL Server on the same server. This will result in performance gains through more efficient processor use and eliminate the impact of network latency between MIIS 2003 and a remote SQL server.
Disk Configuration
Optimize the disk subsystem on the server that hosts the SQL database used by MIIS 2003. Any optimization, such as faster interfaces or high speed storage arrays, will yield improvements. Because the majority of MIIS 2003 transactions involve accessing the database, any performance enhancements applied to the SQL server will result in better MIIS 2003 performance.
Memory Configuration
Servers should be configured with at least 1 GB RAM for each processor. If you will be managing 50,000 or more objects from more than two data sources, then it is recommended that you use 2 GB RAM for each processor.
Network Configuration
If possible, use a 100 Mbps network connection or better. Also, when multiple servers are used to host MIIS 2003 and SQL Server they should be placed on the same network segment. At that speed, network bandwidth should not be a concern. If there are restrictions in your environment, you can use a network as slow as 5 Mbps. However, at that speed, you need to make sure that the network connection is very reliable.
Next
See Also
Concepts
MIIS 2003 Capacity Planning Test Summary - Processor
MIIS 2003 Capacity Planning Test Summary - SQL Server
MIIS 2003 Capacity Planning Test Summary - Disk Performance
MIIS 2003 Capacity Planning Test Summary - Memory
MIIS 2003 Capacity Planning Test Summary - Database Size
MIIS 2003 Capacity Planning Test Summary - Network
MIIS 2003 Capacity Planning - Additional Performance Considerations