Estimate performance and capacity requirements for Visio Services in SharePoint Server 2010

 

Applies to: SharePoint Server 2010

Summary: Test how long it takes to render a data-connected Visio drawing and how many drawings can be processed per second in a given scenario.

This article describes the effects of using Visio Services in Microsoft SharePoint Server 2010 on topologies that are running SharePoint Server 2010.

In this article:

• Test farm characteristics

• Test results

• Recommendations

By using Visio Services, users can view Microsoft Visio Web drawings in SharePoint Server 2010. When Web drawings are connected to external data sources, Visio Services makes it possible for users to update the appearance of Web drawings, based on underlying data changes. For more information about Visio Services, see Visio Services overview (SharePoint Server 2010).

This article discusses the effect of topology on the overall service latency (how long it takes to render a drawing) and throughput (how many drawings can be processed in a second) when a typical mix of drawings is rendered. The test outlined in this article measures performance under recommended user load and maximum user load on an instance of Visio Services that is configured with default settings.

With this information, you can better scale deployments based on latency and throughput requirements. You can either scale up by increasing the capacity of the existing servers or scale out by adding additional servers to the topology. It is important to be aware that the specific capacity and performance figures that are presented in this article differ from the figures in real-world environments. The figures presented are intended to provide a starting point for the design of an appropriately scaled environment. After you complete the initial system design, test the configuration to determine whether the system supports the factors in your environment.

For general information about how to plan and run capacity planning for SharePoint Server 2010, see Performance and capacity management (SharePoint Server 2010).

Test farm characteristics

This section describes the dataset, workloads, hardware settings, topology, and test definitions that were used during the performance and capacity testing of Visio Services.

Dataset

Visio Services capacity and performance depends on, among other factors, the makeup of the Web drawings that are hosted on the service. File-related factors that affect performance are listed in the following table.

Factor	Qualitative effect on performance
Drawing disk size	Larger files sizes increase latency on the SharePoint network.
Drawing complexity	Drawings that contain many complex shapes increase rendering latency and resource usage on the application servers.
Drawing data-connectivity	Visio Web drawings can be either static or data-connected. For static drawings, Visio Services loads drawings and renders directly to the browser. For data-connected drawings, Visio Services performs the additional step of polling the underlying data sources for fresh data and updating the drawing before rendering it to the browser. This additional step increases latency and resource usage on application servers.

To simulate the effect of these factors, files were categorized as shown in the following table.

	Small	Medium	Large
Number of shapes	20	50	100
Typical file size	300 KB	600 KB	900 KB

If drawings were data-connected, these additional categories were used.

	Small	Medium	Large
Number of refreshable elements	10	120	180
Number of rows imported	10	40	80
Percent of data change	50%	50%	50%

The different file categories were then assembled into a test dataset by using the distribution shown in the following table.

Types of file	Percentage
Small static	49
Small data-connected	21
Medium static	14
Medium data-connected	6
Large static	7
Large data-connected	3

Note

Data-connected files were connected to a Microsoft SQL Server data source by using Secure Store Service Authentication. This article does not cover the performance effect of drawings that are connected to other external data sources.

Workload

The following test procedure was used for each performance scenario. Note that this test assumes a farm that is dedicated to Visio Services (no other tests running on SharePoint Server) and that only one user is using the service.

Test ID	Test name	Test description
#1	Rendering a Typical Mix of Visio Web Drawings	From the browser, a simulated user requests Visio Services to render a drawing. Visio Services renders the drawing by using the Portable Network Graphics (PNG) output format and returns it to the user via the browser. If the drawing is data-connected, the drawing is refreshed before it is rendered. Visio Services caches the drawing to reduce latency the next time that it is requested. Steps 1, 2, and 3 are repeated with files randomly selected from the Dataset without any think time between requests.

Green and red zone definitions

For each topology configuration a green zone and red zone user load was determined before throughput tests were run. These configurations are defined in the following table.

Configuration	Definition
Recommended (green zone)	The user load at which the test being run consumes approximately half of the bottlenecking resource. In the case of Visio Services, this is the CPU usage on the front-end Web server (WFE). You should expect the green zone throughput to be sustained for long periods of time in real-world deployments.
Maximum (red zone)	The user load at which maximum throughput was reached for a topology while latency was at minimum. After this point, throughput typically remained stable and latency increased. You should expect the red zone throughput to be tolerated for a short time on a farm, but it should be avoided.

Hardware settings and topology

Lab hardware

To provide trend information, several farm configurations were used for testing with increasing computing capacity. Farm configurations ranged from two to five Web servers plus a single database server that was running SQL Server 2008. Testing was performed by using one client computer that generated all requests. All Web server computers and the database servers were 64-bit.

The following table lists the specific hardware that was used for testing.

	DELL PE 2950	DELL PE 2950	DELL PE R900
Role	WFE	Application server	SQL Server-based server
Processor (CPU)	2pX4 (Xeon L5420@2.5GHz)	2pX4 (Xeon L5420@2.5GHz)	4pX4 (Xeon E7330@2.4GHz)
RAM in gigabytes (GB)	16	16	32
Operating system	Windows Server 2008 R2 Enterprise	Windows Server 2008 R2 Enterprise	Windows Server 2008 R2 Datacenter
Authentication	NTLM	NTLM	NTLM
Storage: Operating System	4x 146 GB, 10 K RPM, RAID 0	4x 146 GB, 10 K RPM, RAID 0	2x 146 GB, 15 K RPM, RAID 1
Storage: Backups	--	--	3x 300 GB, 15 K RPM, RAID 5
Storage: SQL Server data	--	--	9x 300 GB, 15 K RPM, RAID 5
Storage: SQL Server logs	--	--	6x 300 GB, 15 K RPM, RAID 5
Number of instances of SQL Server	0	0	1, SQL Server 2008 SP1 CU6
Number of network adapters	1	1	4
Network adapter speed	1 GB	1 GB	1 GB
Load balancer type	NLB	Not applicable	Not applicable
ULS logging level	Medium	Medium	Medium
Antivirus settings	Microsoft Forefront	Microsoft Forefront	Microsoft Forefront

Lab software

The following table lists the specific software that was installed on the lab computers during testing.

Software	Description
Operating system	Windows Server 2008 R2 Enterprise version 6.1.7600
SQL Server version	SQL Server 2008 version 10.0.2531.0
IIS	Version 7.5.7600.16385
SharePoint Server	SharePoint Server 2010

Topology

The topologies shown in the following diagrams were used to detect scale-out performance trends.

1 WFE x 1 application server x 1 SQL Server data source (1x1x1)

2 WFE x 1 application servers x 1 SQL Server data source (2x1x1)

2 WFE x 2 application servers x 1 SQL Server data source (2x2x1)

3 WFE x 2 application servers x 1 SQL Server data source (3x2x1)

Test results

The following sections show the test results of Visio Services.

After a calibration run that is used to determine the green zone and red zone user loads, the Rendering a Typical Mix of Visio Web Drawings test was run repeatedly. Only the topology was changed, to show its progressive effect on farm performance. The requests per second (RPS) number that is reported in this article is the average RPS of a constant user load test.

Note

All the tests reported on in this article were conducted without think time, which is a natural delay between consecutive operations. In a real-world environment, each operation is followed by a delay as the user performs the next step in the task. By contrast, in this testing, each operation was immediately followed by the next operation, which resulted in a continual load on the farm. This load introduced database contention and other factors that can adversely affect performance.

For information about bottlenecks in Visio Services, see the Common bottlenecks and their causes section later in this article.

Overall scale

The following table summarizes the effect of adding front-end Web servers and application servers to the throughput of Visio Services.

	Recommended (green zone) requests per second	Maximum (red zone) requests per second
1x1x1	97.5	126
2x1x1	172.5	192
2x2x1	195	218
3x2x1	242	266

The following graph shows that the addition of front-end Web servers and application servers increases both green zone and red zone throughput linearly. By slope comparison, the graph of these values also shows that the addition of a front-end Web server has a larger positive effect on the throughput than the addition of application servers. This indicates that front-end Web servers can be a bottleneck for Visio Services deployments.

Throughput vs. Topology

Hardware cost per transaction

The following table highlights the hardware cost of running the Rendering a Typical Mix of Visio Web Drawings test across topologies in the green zone.

Scorecard dashboard	1x1x1	2x1x1	2x2x1	3x2x1
Average RPS (requests/sec)	97.5	172.5	195	242
Average WFE server CPU resources (%)	59.5	54.5	59.8	50.2
Average application server CPU resources (%)	16.4	27.9	17.25	21.5
Failure rate	0.006	0.006	0.01	0.02
90% QoS availability	1	1	1	1
25th percentile latency (seconds)	0.05	0.05	0.05	0.06
Percent time in garbage collector	WFE: 3.55 App: 6.6x10-7	WFE: 3.11 App: 0.00014	WFE: 3.27 App: 0.68	WFE: 3.13 App: 0.125
Number of WFE server crashes	0	0	0	0
Average memory used (bytes)	WFE: 210,728,784 App: 210,172,592	WFE: 664,374,336 App: 810,444,288	WFE: 203,343,584 App: 229,627,536	WFE: 199,905,600 App: 259,536,336
Maximum memory used (bytes)	WFE: 212,664,320 App: 211,578,880	WFE: 719,638,528 App: 1,309,839,360	WFE: 204,537,856 App: 229,969,920	WFE: 200,081,408 App: 262,713,344

The following table highlights the hardware cost of running the Rendering a Typical Mix of Visio Web Drawings test across topologies in the red zone.

Scorecard dashboard	1x1x1	2x1x1	2x2x1	3x2x1
Average RPS (requests/sec)	124	190	216	264
Average WFE CPU resource (%)	73.8	64	71.05	59.9
Average application server CPU resources (%)	18.9	31	18.35	23.0
Failure rate	0.006	0.009	0.009	0.01
90% QoS availability	1	1	1	1
25th percentile latency (seconds)	0.06	0.06	0.07	0.06
Percent time in garbage collector	WFE: 0.000036 App: 0.000074	WFE: 0.00036 App: 0.00014	WFE: 1.54 App: 0.805	WFE: 1.15 App: 0.4
Number of WFE crashes	0	0	0	0
Average memory used (bytes)	WFE: 631,852,288 App: 820,075,648	WFE: 748,467,200 App: 884,640,512	WFE: 659,872,256 App: 511,670,960	WFE: 730,737,301 App: 827,111,104
Maximum memory used (bytes)	WFE: 708,333,568 App: 1,446,760,448	WFE: 787,783,680 App: 1,350,569,984	WFE: 709,833,600 App: 1,319,833,600	WFE: 1,070,150,997 App: 1,450,207,232

Recommendations

This section provides general performance and capacity recommendations. Use these recommendations to determine which hardware, topology, and configuration settings best meet your particular capacity and performance needs.

Hardware recommendations

The basic hardware requirements for Visio Services are the same as those for SharePoint Server 2010, which can be found in the following article: Hardware and software requirements (SharePoint Server 2010).

Scaled-up and scaled-out topologies

To increase the capacity and performance of one of the starting-point topologies, you have two options. You can either scale up by increasing the capacity of the existing servers or scale out by adding additional servers to the topology.

As a rule, Visio Services is especially sensitive to scaling out. When planning to scale out, use the following general rules:

1. When you, use the configurations described earlier, favor increasing WFE servers to increasing application servers. If your computers have performance specifications that are comparable to the specifications in this article, a ratio of three WFE servers to one application server is recommended.

2. Remember that scaling out enables both an increase in throughput and an increase in user load that Visio Services can serve. Use the following graph to gauge the expected throughput on the topology at a particular user load. Notice that throughput tapers off at a certain user load, after which requests are served with increase latency.

Throughput vs. User Load

Settings optimizations

One way to control the performance characteristics of Visio Services is to change the values of its performance-related service settings. The following table discusses the qualitative effect on farm performance of varying these settings.

Setting location	Parameter	Description	Qualitative effect on performance
Central Administration	Maximum Web Drawing Size	The maximum size in MB of a Web drawing that can be rendered.	A larger size limit can lead to reduced throughput and increased latency. A smaller limit can prevent more complex Web drawings from being rendered but has the opposite effect on performance.
	Minimum Cache Age	The minimum number of minutes that a Web drawing is cached in memory. This value is per user per drawing. The interval begins when a user views a Web drawing. That user cannot refresh that Web drawing until the interval expires. The interval begins for other users when they first view the Web drawing.	Smaller values allow for more frequent data refresh operations for users but increase CPU and memory usage on application servers, which reduces throughput and increases latency. A larger value increases the refresh window but has the opposite effect on performance metrics.
	Maximum Cache Age	The number of minutes after which cached Web drawings are purged.	Larger values decrease file I/O and CPU usage on application servers but increase memory usage on the server. Large values reduce latency for drawings that are rendered often. A smaller value has the opposite effect on performance.
	Maximum Recalc Duration	The number of seconds before the data refresh operations time out. This applies only to data-connected Web drawings.	Longer timeouts allow for more complex data-connected Web drawings to be recalculated but use more processing power, reduce throughput, and increase latency. A shorter time-out decreases the complexity of the drawings that can be rendered but has the opposite effect on performance.
Web Part	Force Raster Rendering	Forces the Visio Web Access Web Part to render the drawing as a PNG even though Microsoft Silverlight might be installed.	Choosing to render all drawings by using a raster format reduces drawing visual fidelity but increases throughput slightly. Choosing to render all drawings in XAML increases visual fidelity but reduces throughput slightly.

Common bottlenecks and their causes

The following sections discuss methods for improving farm performance by optimizing service settings and working around common system bottlenecks.

During performance testing, the following bottleneck was revealed. A bottleneck is a condition in which the capacity of a particular constituent of a farm is reached. This causes a plateau or decrease in farm throughput.

Bottleneck	Cause	Resolution
WFE CPU utilization	Because of caching on the application server layer, the throughput of the Visio Services application servers is larger than the front-end Web servers. This causes the WFE layer to be the bottleneck of the system.	Add more front-end Web servers to help reduce the effect of the WFE bottleneck on the overall throughput of Visio Services.

Performance monitoring

To help you determine when you have to scale up or scale out a Visio Services deployment, you can use performance counters to monitor its health. In addition to the performance counters that are included with Windows Server and that measure overall server health, you can use the following list of performance counters to gain a deeper understanding of the specific performance behaviors of Visio Services.

Visio Services performance counters

The following table lists performance counters that monitor key Visio Services application server metrics. All these counters are in the Visio Server: Visio Graphics Service performance counter category.

Logical category	Counter	Measurement
Aggregate counters	Requests Received per Second	Count of requests received per second.
	Request Processing Time	Average processing time of a rendering request in milliseconds.
Drawing rendering pipeline detail	File Retrieval and Parsing Time	Average time, in milliseconds, to retrieve a drawing from the content database and to parse it.
	Server Data Refresh Time	Average time, in milliseconds, to retrieve all external data for a data-connected drawing.
	Server Data Binding Time	Average time, in milliseconds, to update a data-connected drawing.
	Text Data Binding Time	Average time, in milliseconds, to update text in a data-connected drawing.
	Server Rasterization Time	Average processing, in milliseconds, to create a PNG representation of a drawing.

Visio Web Access performance counters

The following table lists performance counters that monitor key Visio Web Access server (front-end Web server) metrics. All these counters are in the Visio Server: Visio Web Access performance counter category.

Logical category	Counter	Measurement
Aggregate counters	Request Processing Time	Average time, in milliseconds, to process a drawing render request from arrival to delivery.
	Requests Received per Second	Count of requests received per second.
	Requests with Errors per Second	Average number of requests that are returned with errors per second.
	Average Content Initial Transmission Time	Average transmission time, in milliseconds, of initial HTML plus Java scripts to client computer.

See Also

Concepts

Configure Visio Services for a BI test environment