A Case of PowerShell Causing memory crunch

While working SharePoint you use PowerShell and get highly attracted to the Pipelining feature and tend to use if more often. But SharePoint holds objects of large size and pipelining these objects would cause some serious performance issues at the server

General Information:

Power Shell script does not do a good job at the garbage collection and fails to remove the objects that have been used during the execution of the script. And the job of writing a clever script ends up in the hands of the script developer to make sure that the script does not cause a memory crunch or CPU crunch on the server. And most importantly a script developer should be aware that the script might end up running on multiple environments of difference scale. Hence the script written should be able to work with huge number of objects.

Issue:

I recently have been hit back very badly where the script prepared and tested in on one of my main DEV environments has caused server out of memory issue on the production. Below is a screenshot of such exception that has occurred during the execution of the script.

Here is the brief structure of the script that I have written and has caused a performance issue on the server in production

 

In the above script the highlighted area is the root cause that has taken a the 8 GB ram on the production server hosting 10361 sites. This line is created and bring the 10361 SPWeb objects to be created with global scope and our brought into server memory prior to actual execution. And also these objects remain in server memory throughout during the execution based on the scope of the dollar bets variable.

On more investigation I identified one way to prevent memory crunch on the server is to go with the Power Shell pipeline. But as you see the logic being executed in the above script cannot be completely moved into a pipeline execution model.

Resolution:

Hence I modified the script such that I only capture the essential and bare minimal things that are needed to create the objects when needed and have the objects removed once we are done with the object is used

 

Major highlights are:

1. I only captured the list of URLs that can be used to get the SPWeb object when need
2. Retrieved the SPWeb object right on demand
3. Removed the SPWeb object when we are done with it

This script work well efficiently and memory did not even touched the half 4 GB of the server memory site.

Lessons learned:

1.  PowerShell script does not have proper garbage collection
2. SharePoint object are huge. Get them or use them on Demand
3. Work with only Simple types(int, string, etc...) as long as possible

Comments

• Anonymous
  June 05, 2014
  I am trying to run this script on our site, do I need to run it every time like a job. If it is, how would you do it? Appreciate your help here