Source Server and Symbol Server Features in Team Foundation Server 2010

Introduction

Jim Lamb wrote a good post about Enabling Symbol and Source Server Support in TFS Build 2010 Beta 1. His post gives a great overview of the requirements to get it working out of the box (it’s very easy!), while this post aims to help the beginner understand how it works, to elaborate on some details of the features, and to identify what customizations are available.

Note that the source server and symbol server features complement each other nicely, but neither requires the other.

Source Server Features

If you are unfamiliar with Microsoft’s Source Server features, it really is just metadata (source index) added to debugging symbols. A debugger with source server support can read the metadata and locate the correct versions of files and download them as needed during a debugging session. The metadata is contained in a “stream” of a PDB file and keeps the server location, the server path, and the version for each source file referenced in the symbols. That’s it. This process of adding metadata to debugging symbols is sometimes referred to as “source indexing”.

The TFS Build 2010 implementation largely follows the lead of ssindex.cmd from the Debugging Tools for Windows package. In fact, some of the tools shipped with Debugging Tools for Windows are redistributed with TFS 2010 so that the Debugging Tools don’t need to be separately installed. This makes enabling source server support extremely easy. All you need to do is set the Index Sources build parameter to true for the default build process template. You can do this for any build definition that uses the default template by changing the default value for the parameter, or you can change it in the Queue Build dialog when you queue a build (for a definition using the appropriate template).

The source indexing is done by the IndexSources activity provided by TFS Build. It takes only one argument called FileList, which is the list of symbol files (PDBs) to add a source index to. By default, all PDBs under the output directory are indexed.

IndexSources uses the current build (the one that is building) and workspace mappings to determine the version control information it needs for the sources, and doesn’t let you change how it uses that information. The limitation here is that if you are building sources that come from multiple workspaces (not recommended) then your source index may be incomplete.

Customizing the Source Index

Consider this a warning that messing with the source index outside of using the IndexSources activity is not supported by TFS.

One of the tools that comes in Debugging Tools for Windows is pdbstr.exe. Its sole purpose is to read and write “streams” from PDB files. One situation you may find yourself in is that your TFS installation is accessible from the web, or from anywhere that requires a different URL than the one that the build machine uses, you may need to read out the source index, modify the TFS URL, and then write the source index back.

You can read the source index to a file using this command:

 pdbstr.exe -r -p:"<pdb path>" -i:"<index file>" -s:srcsrv

Edit the VSTFSSERVER variable value to point to the appropriate external URL. Then write the index back using:

 pdbstr.exe -w -p:"<pdb path>" -i:"<index file>" -s:srcsrv

If any readers have other specific requirements for source indexing, please do let me know and I’ll try to address them.

Symbol Server Features

A symbol server is a Windows network share or folder that has a particular format, managed by symstore.exe, another tool shipped with Debugging Tools for Windows. If you direct a supported debugger to that share to find symbols, it should be able to automatically find the right version for the binaries that you are debugging (providing they exist in the share somewhere).

TFS Build 2010 provides support to add symbols to the symbol server share during a build. This is typically done after source indexing (if source indexing is being used) so that someone debugging a build for which they don’t (directly) have symbols or sources can seamlessly step through code and examine call stacks and variables.

To add symbols to a symbol server share, TFS Build provides the PublishSymbols activity. The activity works out-of-the-box by setting the Path to Publish Symbols build process parameter. You should set it to a UNC path, even if you are publishing to a local machine folder, as this feature was aimed at the “typical” user.

Customizing Symbol Publishing

Your customization options for symbol publishing is pretty wide open. Since PublishSymbols is really just a wrapper around the InvokeProcess activity (also shipped with TFS Build) which calls symstore.exe with the “add” command, you can provide additional command line arguments using the CommandLineArguments argument. Remember, these are additional command line arguments. The StorePath and ProductName activity arguments are required, so you shouldn’t include them in CommandLineArguments. Also, the files to publish are provided to symstore.exe as a temp file that contains a list of files, which is generated by the files provided in the FileList argument. Examples of what you may add to CommandLineArguments are the /3 and /p options. Those are simple examples, refer to symstore.exe documentation on the web for more.

Of course, you could always modify the template and replace PublishSymbols with InvokeProcess in circumstances where PublishSymbols doesn’t give you enough flexibility, but note that PublishSymbols tags the build with information about the symstore.exe transaction so that symbols can be deleted when a build is deleted. You could try to replicate that information yourself, but I won’t detail that here.

The SharedResourceScope Activity

One more thing to note about symbol server publishing is the use of the SharedResourceScope activity in the build process template. The purpose is to make sure that concurrent instances of symstore.exe aren’t adding symbols at the same time, as the tool doesn’t support concurrent access to a symbol server share. SharedResourceScope uses the Team Foundation Server to control access to an arbitrarily named resource, in this case the share. That way, if multiple builds are trying to publish symbols at the same time, the requests are queued and only one will publish at a time, while the others wait (instead of fail due to file access errors or “step on each others’ toes”). PublishSymbols does not care about shared resource locks, but it is contained within the SharedResourceScope, so won’t be executed until the lock is appropriately acquired.

Conclusion

With Team Foundation Server 2010 you can boost the efficiency across your dev and test teams using the provided source server and symbol server features out-of-the-box. Yet, for the experienced teams with more precise requirements, you have a lot of options for customization.