MIIS 2003 Capacity Planning Test Summary - Database Size

Applies To: Windows Server 2003 with SP1

Previous Sections in This Guide

• Introduction to MIIS 2003 Capacity Planning

• MIIS 2003 Capacity Planning Test Summary - Processor

• MIIS 2003 Capacity Planning Test Summary - SQL Server

• MIIS 2003 Capacity Planning Test Summary - Disk Performance

This section discusses data that was gathered during other performance tests and analyzed to determine any relationship between MIIS 2003 database size and the number of objects being managed.

Test Description

While the other tests presented in this capacity planning document were being run, database size was tracked to see how it was affected by the operations being performed. The information presented in this section is an analysis of that data rather than the result of tests performed specifically to generate size-related information.

General Observations and Recommendations

There is no definitive methodology to determine how the MIIS 2003 database will grow during normal operations. Providing estimates that apply to the broad range of MIIS 2003 deployments is difficult. Factors that impact the database size are:

• Number of management agents in the solution: More management agents require more storage for the connector space. The connector space stores a complete subset of the information staged from the connected directory.

• Number of objects in each management agent.

• The number of attributes being staged for each object.

• The size of each attribute.

• The number of multivalued attributes and the size of their values.

• The schedule at which run profiles are executed in the solution: More frequent scheduling of MIIS 2003 run profiles requires more run history data to be collected.

• The frequency with which an organization clears the run history. Run history data is collected over time and if the data is not periodically cleared it can drastically increase database size.

• Maintaining sufficient storage space to be able to clear the run history: MIIS 2003 cleans out the run history information within the MIIS 2003 database within one SQL transaction. The result is that the SQL database log expands in size to contain all the deleted run history information. It does not shrink until the transaction is committed. Thus, you need sufficient additional storage for the database for this expansion. The amount of expansion depends on the size of the MIIS 2003 solution and the amount of run history the organization maintains.

The following section provides examples of the database sizes that where observed during the tests of other MIIS 2003 capacity planning components. Some of these examples can be used to estimate the size of an MIIS 2003 database.

It is recommended that you perform tests in a controlled environment by using a representative sample of data from your production environment to help you estimate your own database size requirements. In doing so, your estimates will be based on actual data and the same type of staging and synchronization operations that will be used in your production environment. Based on these data points, some broad estimates could be made about the projected database size.

Test Scenario

The data presented in this section was gathered from the other tests described elsewhere in this document (specifically, the processor and memory tests). The data was collected from the various platforms used for those tests. For information about the particular server configuration used in each test, see "Server Hardware Configuration" and "MIIS 2003 Configuration" in "MIIS 2003 Capacity Planning Test Summary - Processor" and "MIIS 2003 Capacity Planning Test Summary - Memory" in this document.

Test Results

Data that pertains to the database size from two test scenarios is presented here. The first set of data is from tests that determined the processor requirements for MIIS 2003. This scenario consisted of two management agents that imported data into the metaverse and then exported information to a third management agent. During these tests, the database growth was measured.

The second set of data was gathered from the memory tests. It provides an example of what happens to the database size when the number of management agents that process data increases. That scenario used ten management agents to import and synchronize data into an MIIS 2003 database. Database growth was also monitored during these tests.

Collectively, the data provided below indicates both total size and the rate the database grew.

Database Sizing Based on Processor Testing Scenario

For a complete description of the server configuration used to generate this data, see the "Server Hardware Configuration" and "MIIS 2003 Configuration" information in "MIIS 2003 Capacity Planning Test Summary - Processor" in this document.

Table 25: Physical Database File Size Growth (File level measured in MB)

Management Agent	Operation	10,000 Objects	50,000 Objects	100,000 Objects	200,000 Objects	500,000 Objects
Empty Database		4	4	4	4	4
Active Directory	Full Import	4	4	4	4	4
TXT	Full Import	40	169	329	641	1,662
SQL	Full Import	71	329	641	1,374	3,564
Active Directory	Full Sync	71	329	641	1,374	3,564
TXT	Full Sync	139	705	1,374	2,945	7,640
SQL	Full Sync	169	853	1,662	3,240	8,404
Active Directory	Export	169	853	1,662	3,240	8,404
Active Directory	Delta Import	186	853	1,829	3,564	8,404
SQL	Delta Import (20% changed records)	186	853	1,829	3,564	8,404
SQL	Delta Sync	186	853	1,829	3,564	8,404
Active Directory	Export	186	853	1,829	3,564	9,245
Active Directory	Delta Import	186	853	1,829	3,564	9,245
SQL	Delta Import (10% deleted records)	186	853	1,829	3,564	9,245
SQL	Delta Sync	186	938	1,829	3,564	9,245
Active Directory	Export	186	938	1,829	3,564	9,245
Active Directory	Delta Import	186	938	1,829	3,564	9,245

Note

These figures do not include any additional increase in size due to run history data that would normally be generated by run profile activity in a production environment. In the test environment, the run profiles were only processed a limited number of times. In a production environment, they might get processed multiple times daily.

The database sizes are based upon the physical MDF file size on the file system. This is why the file grows to a certain point and then remains constant. This is due to the extents that SQL Server performs. By default this is a 10% growth in database size. In the case of these tests, the data added to the database did not warrant another extent.

Table 26: Description of run profile operations used to generate the data in Table 25.

Management Agent	Operation
Active Directory	Empty Active Directory staged to Active Directory management agent connector space
TXT	Variable number of user staged from TXT management agent (see columns in Table 16 above)
SQL	Variable number of user staged from SQL management agent (see columns in Table 16 above)
Active Directory	Synchronization of Active Directory management agent connector space
TXT	Synchronization of TXT management agent connector space
SQL	Synchronization of SQL management agent connector space
Active Directory	Export the number of users to Active Directory
Active Directory	Delta Import from Active Directory management agent
SQL	Delta Import of 20% change in published number of objects
SQL	Delta Synchronization of changed records
Active Directory	Export the changes to Active Directory
Active Directory	Delta Import from Active Directory management agent
SQL	Delta Import of 10% deletions in the published number of objects
SQL	Delta Synchronization of the deleted records
Active Directory	Export the deletions to Active Directory
Active Directory	Delta Import from Active Directory MA

Database Growth Based on Multiple Management Agent Testing

During these tests, ten identical text sources were sequentially imported and synchronized into an MIIS 2003 server. All the data within each of the management agents were identical and thus joined 100% to other records within the metaverse. During these tests, the file size of the database, as reported by SQL Server and the file system, was recorded.

For a complete description of the server configuration used to generate this data, see "Server Hardware Configuration" and "MIIS 2003 Configuration" in the "MIIS 2003 Capacity Planning Test Summary - Memory" in this document.

Table 27: Ratio of growth during staging, based on database size compared to initial database size. (Used to plot Figure 16)

Objects	MA1	MA2	MA3	MA4	MA5	MA6	MA7	MA8	MA9	MA10
10,000	1.00	1.95	2.92	3.83	4.86	5.78	6.68	7.61	8.74	9.67
50,000	1.00	2.00	2.99	4.00	5.03	5.98	6.94	7.98	9.10	10.06
100,000	1.00	1.99	2.99	4.02	5.04	5.99	6.95	8.01	9.11	10.07
200,000	1.00	2.01	3.01	4.04	5.06	6.02	6.97	8.03	9.16	10.12

Observations

• We can see a linear relation to the growth of the database, which was expected because all the sources that were imported into the database were equal in size.

• Note that with the run history information and the extra management agent data, the database doubles in size after each run profile staging operation.

Table 28: Ratio of growth during synchronization, based on database size compared to initial database size. (Used to plot Figure 17)

Objects	MA1	MA2	MA3	MA4	MA5	MA6	MA7	MA8	MA9	MA10
10,000	1.00	1.02	1.04	1.05	1.07	1.09	1.11	1.13	1.15	1.16
50,000	1.00	1.02	1.04	1.06	1.08	1.10	1.12	1.14	1.16	1.18
100,000	1.00	1.02	1.04	1.06	1.08	1.11	1.13	1.15	1.17	1.19
200,000	1.00	1.02	1.04	1.06	1.08	1.10	1.13	1.15	1.17	1.19

Observations

• Again a linear growth of the database is apparent, which was expected because all the sources that were synchronized into the database were equal in size and joined perfectly to all existing data.

We also notice that the cost in terms of database size to facilitate the linking between is quite small. Because all records in all connector spaces were identical, the metaverse itself did not grow after the initial projection. Upon closer examination, we found that most of the information was written to the “mms_csmv_link” table, which contains linkage between the various connector spaces and the metaverse objects. The rest of the information was written to the run history tables in the MIIS 2003 database.

Table 29a: Database growth as reported by both SQL Server and the server file system during processing of 10,000 and 50,000 objects (listed in Kbytes)

MA	Operation	10,000 SQL DB	MDF DB	50,000 SQL DB	MDF DB
Empty Database		904	2,359	904	2,359
TXT01	Import	28,192	33,227	135,728	153,289
TXT02	Import	55,024	59,048	271,632	298,910
TXT03	Import	82,216	86,442	405,424	437,649
TXT04	Import	108,032	115,081	542,664	582,550
TXT05	Import	137,128	153,289	682,360	704,905
TXT06	Import	162,816	185,532	811,736	853,017
TXT07	Import	188,448	204,079	942,608	1,032,192
TXT08	Import	214,624	224,526	1,083,528	1,135,411
TXT09	Import	246,408	271,712	1,234,616	1,373,897
TXT10	Import	272,560	298,910	1,364,992	1,511,326
TXT01	Sync	299,488	328,794	1,504,248	1,662,452
TXT02	Sync	304,400	328,794	1,534,200	1,662,452
TXT03	Sync	310,576	328,794	1,564,328	1,662,452
TXT04	Sync	315,552	328,794	1,595,320	1,662,452
TXT05	Sync	321,720	361,693	1,623,848	1,828,717
TXT06	Sync	325,304	361,693	1,654,912	1,828,717
TXT07	Sync	331,320	361,693	1,687,128	1,828,717
TXT08	Sync	337,160	361,693	1,716,312	1,828,717
TXT09	Sync	343,488	361,693	1,744,944	1,828,717
TXT10	Sync	347,272	361,693	1,773,856	1,828,717

Table 29b: Database growth as reported by both SQL Server and the server file system during processing of 100,000 and 200,000 objects (listed in Kbytes)

MA	Operation	100,000 SQL DB	MDF DB	200,000 SQL DB	MDF DB
Empty Database		904	2,359	904	2,359
TXT01	Import	264,320	298,910	518,864	582,550
TXT02	Import	527,168	582,550	1,042,680	1,135,411
TXT03	Import	791,160	853,017	1,562,704	1,662,452
TXT04	Import	1,061,400	1,135,411	2,093,984	2,212,823
TXT05	Import	1,331,976	1,373,897	2,623,080	2,945,384
TXT06	Import	1,582,112	1,662,452	3,121,848	3,239,969
TXT07	Import	1,836,888	2,011,628	3,615,928	3,920,429
TXT08	Import	2,116,352	2,212,823	4,166,984	4,312,465
TXT09	Import	2,408,968	2,677,604	4,752,512	5,218,173
TXT10	Import	2,661,424	2,945,384	5,253,304	5,740,036
TXT01	Sync	2,940,536	3,239,969	5,821,616	6,314,066
TXT02	Sync	3,002,392	3,239,969	5,940,104	6,314,066
TXT03	Sync	3,062,384	3,239,969	6,061,168	6,314,066
TXT04	Sync	3,126,416	3,239,969	6,184,776	6,314,066
TXT05	Sync	3,185,096	3,563,979	6,301,720	6,945,505
TXT06	Sync	3,249,712	3,563,979	6,424,568	6,945,505
TXT07	Sync	3,315,240	3,563,979	6,551,032	6,945,505
TXT08	Sync	3,375,392	3,563,979	6,669,616	6,945,505
TXT09	Sync	3,431,872	3,563,979	6,787,696	6,945,505
TXT10	Sync	3,492,112	3,920,429	6,906,944	7,640,056

Observations

• Because of the way that the MDF file gets extended by SQL server, there is usually a huge jump in growth at various points.

Next

• MIIS 2003 Capacity Planning Test Summary - Database Size

See Also

Concepts

Introduction to MIIS 2003 Capacity Planning
MIIS 2003 Capacity Planning Test Summary - Processor
MIIS 2003 Capacity Planning Test Summary - SQL Server
MIIS 2003 Capacity Planning Test Summary - Disk Performance
MIIS 2003 Capacity Planning Test Summary - Memory
MIIS 2003 Capacity Planning Test Summary - Network
MIIS 2003 Capacity Planning - Additional Performance Considerations