An adventure in the sccmcontentlib - single instance store

Sometimes I have nightmares which are work related. One such nightmare was about Configuration Manager 2012 and it prompted me to write this blog post. In my dream, the telephone rang and it was my Configuration Manager 2012 Distribution point. It kindly told me that one of my packages disappeared for no reason. Geez I need a vacation.

With SCCM 2007, content is stored on a distribution point in package ID folders. If you knew the package ID, you could browse to the file share and install/run manually.

This option is more complicated now in Configuration Manager 2012. The files are not just stored on a drive in a manner you could just copy and execute.

The following blog post is about the new Content Libraryand my attempts to reconstitute a package from the Single Instance Store.

By default, the content is single storage and browsing to a DP does not store all the files in a package id folder where you could find an install and execute it. You can enable the older method which will create a separate folder share. This will require double the disk space! Obviously, this is a pain point for those who use the distribution points this way as a mitigation step in a crisis.

To understand how the Content Library works in Configuration Manager 2012, I decided to find a way to solve this pain point.

This functional diagram is my review of the single content library:

 

Content Layer

In the Content Layer, individual files are sent and only stored once in a single storage location. For example, there might be three packages that need to be distributed in a Configuration Manager 2012 infrastructure. All three packages contain the same file Psexec.exe with the same md5\sha1\sha2 hash value.

Why send the file and store it in three places?

Great efficiencies are gained with the new shared single storage Content Library in Configuration Manager 2012. The file is hashed, given an ID and sent only once. Each package is stored as pointers to the necessary files. This means that packages share files if they are the same. This saves on network usage and file storage requirements. Sounds awesome; doesn’t it!

Configuration Manager Layer

Here is an example:

I created two packages:

1. 1.      Filezilla – package id HH00005
2. 2.      Notepad ++ - Package id HH00006

Each has psexec.exe, one of my favorite sysinternal tools. Note you can get it here at https://live.sysinternals.com

Next I “Distribute Content” in the CM2012 console to my distribution point.

Distribution Point Layer

When content is sent to the distribution point, it is stored by default on the largest available drive in a folder, sccmcontentlib. During the install you can set this location, however, you cannot configure the drive settings in the distribution point properties after installation completes.

The following folders get created:

• ·        Datalib = Data Library
• ·        Filelib = File Library
• ·        Pkglib = Package Library

DataLib

For each package that you distribute a corresponding package+id.version number folder and file will be created. For patches the folder will appear as the patch guid and application folders will appear as Content_guid

Now let’s look at the package HH00005.1 (Filezilla)

Each DataLib folder contains .ini file which includes important information for the actual file.

• ·        Attributes
• ·        Size
• ·        TimeModified
• ·        Hash

 Viewing the psexec.ini file in the HH00005.1 package folder:

 

A Hash value of F8DB… is found. The first four entries of this hash are important. This is the locater folder name for the actual binary file in the pkgLib folder.

Note

The default hashing algorithm for content is SHA-256

To find the file:

Filelib

The folder names have meaning. Each corresponds to a hash value in the FileLib folder. In the example of HH000005.1, psexec.exe.ini, the value was F8DB

Viewing the F8DB folder:

The hash value .ini contains a list of the type and references of which packages use this file. Notice that there are two packages that are listed. This is single instance storage. Any package/program/content/update that has this exact hash file requirement will use this file and location.

The actual file will be the hash with no extension. Copying and renaming the file to psexec.exe, I verified that is the psexec file:

SHA-256 hash value matches.

Now back to the HH00005 package. It contains the setup for FileZilla and psexec.

Note that the psexec.exe.ini file points to the F8DB filelib folder as I demonstrated above.

To summarize the Content Library, Single Content Storage and this review:

Both HH00005 and HH00006 CM2012 packages contain the file psexec.exe. The file is only sent once to the Content Library and stored as single instance. A SHA-256 hash value is used to track the unique file and the first four digits are used as a locator. When a client requests the download the files are assembled from the Content Library via single instance storage.

Now suppose you want to reconstitute an entire package from the sccmcontentlib folder. I created a basic vb.net application to assist me in this exercise.

Following each PkgLib file to the DataLib, I was able to regenerate all the files from the FileLib for any given package.

I hope my blog posting was helpful in providing an understanding the Content Library and the Single Instance Storage in Configuration Manager 2012.

Comments

• Anonymous
  January 01, 2003
  Excellent walk through of how the single-instance store works. Thanks!

• Anonymous
  January 01, 2003
  Sure . I've since moved it to PowerShell but I will clean both up and post it back to this link.
  

• Anonymous
  January 01, 2003
  AGREE with the previous comments, Howard! THis is the best I've yet to find that explains this quite complex process in detail as you've done. Thans much for this!!!

• Anonymous
  January 01, 2003
  Any chance we can get the VB app you made ?

• Anonymous
  January 01, 2003
  Between Hodads and those darn breakfast burritos - I'll need more than a vacation. Love the weather in San Diego!! Looking forward to working the summer with you.

• Anonymous
  January 01, 2003
  hi,
  thanks for this post , for me its become clearer .Any news about vb or powershell app ?

• Anonymous
  May 31, 2012
  Howard, You are just finishing up that gig in Houston and are back working with me in San Diego. You are having a vacation. We need to make a trip to Hodads to get rid of these CM 2012 nightmares and get you dreaming about food.

• Anonymous
  June 01, 2012
  I can't believe you're not going to share your app with us . C'mon, pretty please!

• Anonymous
  September 26, 2012
  I have some packages were no packageid.ini files is listed on the distribution point. What can be the cause or is it by design? Thanks

• Anonymous
  July 09, 2013
  Do you know if SCCM 2012 provides an API to "recompile" this?

• Anonymous
  August 06, 2013
  The comment has been removed

• Anonymous
  August 09, 2013
  Howard....this is an excelllent post on explaining how the content lib sis functions.  Great Job!

• Anonymous
  September 13, 2013
  I was really hoping this would be true binary / block-level single-instance storage.   I don't see how this really would help much at all.  saving a few hundred kilobytes on a copy of psexec isn't much.  You'd have to have an extraordinarily large SCCM file structure to have enough identical files for it to really make a difference, and I'd still say it's going to be very little percentage. :(

• Anonymous
  June 09, 2014
  ~ Bindusar Kushwaha Hi everyone, here in product support we regularly see issues relating to how the

• Anonymous
  July 09, 2014
  Great post Howard.. very useful..
  Hey.. is there any chance that you share the vb.net application to reconstitute packages with us?

• Anonymous
  August 13, 2014
  The comment has been removed

• Anonymous
  March 24, 2015
  Anyone finding an alternative link to the tool (VB, powershell, etc.) which is presented above, please post it... I'm having no luck locating something...

• Anonymous
  March 25, 2015
  The comment has been removed

• Anonymous
  March 25, 2015
  hi JMO
  hi I asked my PFE to find it. I will send it as soon as I
  laurent

• Anonymous
  June 30, 2015
  Does anyone have a link they can share to the powershell or vb tool?

• Anonymous
  October 19, 2015
  The comment has been removed

• Anonymous
  November 14, 2015
  I tried this tool but cannot find any link to download vb script/powershell
  
  Thanks,
  Kim Aller,
  https://techsupportphonenumber.net/mcafee-contact-phone.html