Why SharePoint 2013 Cumulative Update takes 5 hours to install?
I recently went through the process of applying the latest SharePoint 2013 Cumulative Update on my multi server SharePoint 2013 Farm. To my surprise, I found the total time to apply the patch (install the bits) took over 5 hours per SharePoint Server. This time doesn’t include running PSConfig which is considered phase 2. My SharePoint 2013 farm is hosted by a Windows 8 Hyper V environment where my VM’s meet the minimum hardware requirements for memory etc. I found that as I started to gracefully terminate other SharePoint services running, the time of subsequent patch runs started to drop tremendously. I suspect the following:
Some of these processes like App Fabric, Search(node runner), etc.. are taking enough CPU time, threads associated with Windows Installer gets put into a longer line waiting for the most popular roller coaster ride which happens to be the CPU. Note: Threads are treated the same as everyone else in terms of thread priority.
Another way of saying this:
By having x # of additional processes running, you increase the # of threads vying for CPU time by x much. Assuming each process threads are at the same priority, each thread will evenly get a slide of Processor time to execute.
For Example: Let’s assume I have 2 processes (process A) and (process B). Process A has 10 run-able threads while Process B has 3 run-able threads. Assuming these 13 threads are at the same priority, each thread would theoretically receive one-thirteenth of CPU time.
I wrote a Power Shell script to automate and speed up the install of a Cumulative Update on SharePoint 2013. This Power Shell script improved my patch time from 5 hours to 30 minutes. The patch performs the following steps:
1. Disable the IISAdmin and SPTimerV4 service
2. Shut down IIS Admin and Timer Services if they are running
3. Give you the option to Pause the Search Service Application (see search notes below)
4. Stop Search Services (see search notes below)
5. Install the patch in passive mode (No user interaction required but will witness the patch install in the UI)
Note: Power Shell should remain open in the background while patch is running
6. Upon completion of the patch, the Power Shell script, services in step 1 are set to Automatic
7. Starts up IIS Admin and Timer Services
8. Starts up Search Services
9. Resume the Search Service Application if it was paused
10. Finally, the script will display the Start Time and End Time for patch install
Search Notes:
- This script will only stop Search SharePoint Services if they are running
- This script will only pause the Search Service Application upon user input
- This script will only start Search SharePoint Services if they were stopped
- This script will only resume the Search Service Application if it was paused
Important: I recommend pausing the Search Service Application for several reasons but primarily so that Search services aren’t being shutdown during a Full, Incremental, or Continuous Crawl. I made this an option because a SharePoint Admin can run the patch simultaneously on multiple SharePoint servers and may prefer to manually pause/resume the Search Service Application.
Instructions for Running the Power Shell Script
In order to run the script, save the flow script to a text editor like notepad. Save the file with a .ps1 extension to a folder on the SharePoint Server. Drop the SharePoint cumulative update in the same folder as the script. In my example, I created a scripts folder where I dropped the powershell script and the cumulative update.
For Example:
Important: Only one update should be placed here and the file extension should be .exe.
Finally, to run the script from c:\scripts using SharePoint Management Shell. This script is tested on Windows 2012 running SharePoint 2013.
Example Screenshot – during patching it looks like:
Example Screenshot – Patching is complete!
Updated Script 4/4 1:41 AM (CST). Thanks to Scott Manning for script validation efforts with appfabric.
Updated Script 4/2 2:10 PM (CST). Thanks to Jon Waite for script validation efforts.
Updated Script 8/31 10:00 AM (CST). Corrected an issue with starting script and removed appfabric shutdown/start up failed logic. Thx for the feedback!
Script is below, start copying and enjoy!
<# ==============================================================
//
// Microsoft provides programming examples for illustration only,
// without warranty either expressed or implied, including, but not
// limited to, the implied warranties of merchantability and/or
// fitness for a particular purpose.
//
// This sample assumes that you are familiar with the programming
// language being demonstrated and the tools used to create and debug
// procedures. Microsoft support professionals can help explain the
// functionality of a particular procedure, but they will not modify
// these examples to provide added functionality or construct
// procedures to meet your specific needs. If you have limited
// programming experience, you may want to contact a Microsoft
// Certified Partner or the Microsoft fee-based consulting line at
// (800) 936-5200 .
//
// For more information about Microsoft Certified Partners, please
// visit the following Microsoft Web site:
// https://partner.microsoft.com/global/30000104
//
// Author: Russ Maxwell (russmax@microsoft.com)
//
// ---------------------------------------------------------- #>
###########################
##Ensure Patch is Present##
###########################
$patchfile = Get-ChildItem | where{$_.Extension -eq ".exe"}
if($patchfile -eq $null)
{
Write-Host "Unable to retrieve the file. Exiting Script" -ForegroundColor Red
Return
}
########################
##Stop Search Services##
########################
##Checking Search services##
$srchctr = 1
$srch4srvctr = 1
$srch5srvctr = 1
$srv4 = get-service "OSearch15"
$srv5 = get-service "SPSearchHostController"
If(($srv4.status -eq "Running") -or ($srv5.status-eq "Running"))
{
Write-Host "Choose 1 to Pause Search Service Application" -ForegroundColor Cyan
Write-Host "Choose 2 to leave Search Service Application running" -ForegroundColor Cyan
$searchappresult = Read-Host "Press 1 or 2 and hit enter"
Write-Host
if($searchappresult -eq 1)
{
$srchctr = 2
Write-Host "Pausing the Search Service Application" -foregroundcolor yellow
Write-Host "This could take a few minutes" -ForegroundColor Yellow
$ssa = get-spenterprisesearchserviceapplication
$ssa.pause()
}
elseif($searchappresult -eq 2)
{
Write-Host "Continuing without pausing the Search Service Application"
}
else
{
Write-Host "Run the script again and choose option 1 or 2" -ForegroundColor Red
Write-Host "Exiting Script" -ForegroundColor Red
Return
}
}
Write-Host "Stopping Search Services if they are running" -foregroundcolor yellow
if($srv4.status -eq "Running")
{
$srch4srvctr = 2
set-service -Name "OSearch15" -startuptype Disabled
$srv4.stop()
}
if($srv5.status -eq "Running")
{
$srch5srvctr = 2
Set-service "SPSearchHostController" -startuptype Disabled
$srv5.stop()
}
do
{
$srv6 = get-service "SPSearchHostController"
if($srv6.status -eq "Stopped")
{
$yes = 1
}
Start-Sleep -seconds 10
}
until ($yes -eq 1)
Write-Host "Search Services are stopped" -foregroundcolor Green
Write-Host
#######################
##Stop Other Services##
#######################
Set-Service -Name "IISADMIN" -startuptype Disabled
Set-Service -Name "SPTimerV4" -startuptype Disabled
Write-Host "Gracefully stopping IIS W3WP Processes" -foregroundcolor yellow
Write-Host
iisreset -stop -noforce
Write-Host "Stopping Services" -foregroundcolor yellow
Write-Host
$srv2 = get-service "SPTimerV4"
if($srv2.status -eq "Running")
{$srv2.stop()}
Write-Host "Services are Stopped" -ForegroundColor Green
Write-Host
Write-Host
##################
##Start patching##
##################
Write-Host "Patching now keep this PowerShell window open" -ForegroundColor Magenta
Write-Host
$starttime = Get-Date
$filename = $patchfile.basename
$arg = "/passive"
Start-Process $filename $arg
Start-Sleep -seconds 20
$proc = get-process $filename
$proc.WaitForExit()
$finishtime = get-date
Write-Host
Write-Host "Patch installation complete" -foregroundcolor green
Write-Host
##################
##Start Services##
##################
Write-Host "Starting Services Backup" -foregroundcolor yellow
Set-Service -Name "SPTimerV4" -startuptype Automatic
Set-Service -Name "IISADMIN" -startuptype Automatic
##Grabbing local server and starting services##
$servername = hostname
$server = get-spserver $servername
$srv2 = get-service "SPTimerV4"
$srv2.start()
$srv3 = get-service "IISADMIN"
$srv3.start()
$srv4 = get-service "OSearch15"
$srv5 = get-service "SPSearchHostController"
###Ensuring Search Services were stopped by script before Starting"
if($srch4srvctr -eq 2)
{
set-service -Name "OSearch15" -startuptype Automatic
$srv4.start()
}
if($srch5srvctr -eq 2)
{
Set-service "SPSearchHostController" -startuptype Automatic
$srv5.start()
}
###Resuming Search Service Application if paused###
if($srchctr -eq 2)
{
Write-Host "Resuming the Search Service Application" -foregroundcolor yellow
$ssa = get-spenterprisesearchserviceapplication
$ssa.resume()
}
Write-Host "Services are Started" -foregroundcolor green
Write-Host
Write-Host
Write-Host "Script Duration" -foregroundcolor yellow
Write-Host "Started: " $starttime -foregroundcolor yellow
Write-Host "Finished: " $finishtime -foregroundcolor yellow
Write-Host "Script Complete"
Thanks,
Russ Maxwell, MSFT