The Ramblings of Two Microsoft .NET Developers, TFS, and Visual Studio ALM Guys --- "Yes, we are both named Ed."

Source Server and Symbol Server Support in TFS 2010



As Jim Lamb announced in June 2009, TFS 2010 introduces support for Source Server and Symbol Server as part of the default automated build process template. This is a really key feature addition but I have found that many developers ask about why it would be so important and why it would help them. Ultimately, we are starting to have more and more tools that need access to the symbol file information and the original source code that was used for compilation. For example, some of the tools that come to mind are:

By setting up Source Server and Symbol Server support during your build process, you’ll be able to work with assemblies & executables that come from the build servers and still use tools that need information from them.

What are Symbols?

imageJohn Robbins has an excellent blog post to get started about learning what symbols are titled: “PDB Files: What Every Developer Must Know.” I highly recommend you take a moment to read through it.

So to summarize from John’s article, the symbol files are the .PDB files that match a particular assembly and contain important information that’s necessary for debugging tools. Specifically for .NET assemblies, the symbol files include:

  • Source File Names and Line Numbers
  • Local Variable Names

He also reminds us one very important statement about symbol files: “PDB files are as important as source code!” That is absolutely true! I cringe any time I hear from a developer that says “oh, those .PDB files take up so much space so I’m going to delete them.” Ouch – The sad thing is those are developers that keep people like John in business whenever they run into problems in production. Smile Save yourself some time, money, and effort and keep your symbol files around. Not to say that John doesn’t earn every penny but I’m sure his life is much better whenever you do have your symbols!

This is exactly where Symbol Server helps out. Essentially, the Symbol Server is a central location for your company that keeps the .PDB files for you. Therefore, you can install your application (without symbols) that was compiled from a build server and whenever you want to use a debugging tool like Visual Studio, it will know how to contact the Symbol Server location to get the matching set of symbols. More about how to configure Visual Studio to look for a Symbol Server further down in this blog post.

John also mentions how to manually perform the steps necessary for completing the loop with Source Server and Symbol Server. Thankfully, since you are using TFS 2010 Build, you don’t have to go through those steps. The functionality is included in the default build build process template (but not the Upgrade Template).


Aside: If you are performing obfuscation using your favorite .NET obfuscation utility, you will want to make sure you produce symbol files that match the newly created assemblies. This is because the variable names and other information change by the obfuscator. What I will normally do will do is keep both the original assemblies with their matching symbol files in addition to the obfuscated assemblies with match symbol files. I store the artifacts for the obfuscated assemblies in a sub-folder called “Obfuscated.”

imageimage


How to Setup Symbol Server

A common misconception about Symbol Server is that you actually have to set up a server and install the Symbol Server software. Not at all! All you have to do is setup a file share on another server. If you are using my suggestion about using friendly DNS names with TFS, you might extend that for the symbol server as well:

\\symbols.contoso.local\Symbols

On my particular demonstration machine, I have a local file share that contains some of the symbols that were published from my TFS 2010 Builds:

image

How to Configure Build to Index for Source Server and Publish to Symbol Server

Configuring the build definition to use the new Symbol Server location, couldn’t be easier. Open up the build definition editor and navigate to the Process tab. There, you will see all of the process parameters. If you are using the default build process template then you will find the Source Server and Symbol Server settings underneath the “2. Basic” category as shown below.

image

The build process will then do all the work for you!

Source Server Indexing

What actually happens when the build process is actually running it’s Source Server indexing? Let me first start by discussing the problems with symbols that come from a build server (or another machine.) One of the pieces of information that is stored inside of the symbol file is the location of the original source file that was used for compilation into the assembly you are debugging. This can be a problem because for my particular case, the local location of the source code file on the build server is:

C:\LocalBuilds\1\2\Sources\Source\Calculator\Calculator\Form1.cs at version 32 from the MAIN branch

Not only do you to have put all of the source files in the same exact spot but you would have to get it from the right branch and even the exact same changeset version from the TFS version control repository. That’s a lot of manual work… This is where the indexing for Source Server helps you out. You’ll also notice that if you are producing symbols from your obfuscation utility, those can indexed for Source Server support as well.

image

When the TFS 2010 Build runs the source indexing for Source Server, it writes an alternate stream of information in the symbol files that will provide the following information for each source file:

  • Source Control provider’s information and the command-line utility to use to get the file (In our case that would be using tf.exe)
  • Full TFS Version Control Repository Server Path including the branch name
  • Version

The default build process template uses the srctool.exe command-line utility first to list all of the local source file locations that are stored in the symbol file. Then, it generates a temporary file that contains the exact alternate stream information for Source Server. The Source Server stream is named srcsrv. Finally, the build process uses the pdbstr.exe command-utility to add that stream information to write the relevant information. If you are ever curious about what that srcsrv stream actually contains, you can run this command-line utility:

C:\Builds\Calculator MAIN\Calculator MAIN_11.02.11.06\Debug\Obfuscated>pdbstr.exe –r -p:Calculator.pdb -s:srcsrv
SRCSRV: ini ------------------------------------------------
VERSION=3
INDEXVERSION=2
VERCTRL=Team Foundation Server
DATETIME=Fri Feb 11 00:41:58 2011
INDEXER=TFSTB
SRCSRV: variables ------------------------------------------
TFS_EXTRACT_CMD=tf.exe view /version:%var4% /noprompt "$%var3%" /server:%fnvar%(%var2%) /console >%srcsrvtrg%
TFS_EXTRACT_TARGET=%targ%\%var2%%fnbksl%(%var3%)\%fnfile%(%var5%)
SRCSRVVERCTRL=tfs
SRCSRVERRDESC=access
SRCSRVERRVAR=var2
VSTFSSERVER=http://localhost:8080/tfs/DefaultCollection
SRCSRVTRG=%TFS_extract_target%
SRCSRVCMD=%TFS_extract_cmd%
SRCSRV: source files ---------------------------------------
C:\LocalBuilds\1\2\Sources\Source\Calculator\Calculator\Form1.cs*VSTFSSERVER*/Calculator/MAIN/Source/Calculator/Calculator/Form1.cs*32*Form1;C32.cs
C:\LocalBuilds\1\2\Sources\Source\Calculator\Calculator\Form1.Designer.cs*VSTFSSERVER*/Calculator/MAIN/Source/Calculator/Calculator/Form1.Designer.cs*30*Form1.Designer;C30.cs
C:\LocalBuilds\1\2\Sources\Source\Calculator\Calculator\Program.cs*VSTFSSERVER*/Calculator/MAIN/Source/Calculator/Calculator/Program.cs*30*Program;C30.cs
C:\LocalBuilds\1\2\Sources\Source\Calculator\Calculator\Properties\Settings.Designer.cs*VSTFSSERVER*/Calculator/MAIN/Source/Calculator/Calculator/Properties/Settings.Designer.cs*11*Settings.Designer;C11.cs
SRCSRV: end ------------------------------------------------

Publishing to Symbol Server

Publishing the symbols is the easier part of it. Essentially, the default build process template calls the symstore.exe add utility to publish the symbol files to the specified symbol server path. Additionally, there is some metadata added for the build information in TFS that will specify that symbols were published. This will be useful whenever the build retention policies kick in which we’ll cover further down.

Configuring Visual Studio to Use Symbol Server and Enabling Source Server Support

The next step is for each of the developers to configure Visual Studio 2010 to look for symbols if they aren’t found in the symbol server location for the company. You can get to it by going to Tools –> Options and then the Debugging –> Symbols options pages as shown below. Other debugging tools have similar options.

SNAGHTML23f64f5

The next thing you will want to do is to enable source server support in Visual Studio. You can do that by going to the Debugging –> General options tab as shown below.

SNAGHTML243d0c1

Now, just start using your debugging tool and in my case I have attached my Visual Studio Debugger to the process of my application that came from the build drop folder. Visual Studio gives me a small warning before it attempts to grab the source code from the TFS Version Control repository as shown below. You can see the exact command-line utility including arguments that is used by the debugger to retrieve the correct version of the file. Pure magic…

SNAGHTML226bcf3

Update:  (2/14/2011) John Robbins has helped out by letting us know how we can disable this really annoying Source Server security dialog any time the debugger wants to get something from Source Server.  Thanks John!


Aside: If you notice, in my situation I have a particular problem. Since the TFS 2010 Build services are installed on the same machine as my application tier on my laptop, the default configuration for the build service to connect to TFS used http://localhost. Sad smile That’s not going to be good whenever I have another developer start debugging using the assembly from my build server and the symbols. Their Visual Studio Debugger instance will try to hit localhost on their machine (where the source doesn’t exist).

For this reason, it’s important to make sure when you are configuring the build service to use the fully-qualified friendly DNS name for your application tier server. (Check out the blog post that’s linked to find out more information about this topic).

image


How Does Visual Studio Know Which Symbols Match for the Executable?

You have to always have symbol files that exactly match the assemblies you are debugging. How does Visual Studio know this though? There is actually a GUID that is embedded to both the assembly and the symbol file. You can find out what that GUID is by running the DUMPBIN command-line utility as shown below.

C:\Builds\Calculator MAIN\Calculator MAIN_11.02.11.06\Debug>dumpbin Calculator.exe /HEADERS

Microsoft (R) COFF/PE Dumper Version 10.00.31118.01
Copyright (C) Microsoft Corporation. All rights reserved.


Dump of file Calculator.exe

Debug Directories

Time Type Size RVA Pointer
-------- ------ -------- -------- --------
4D54CC09 cv 69 00003864 1A64 Format: RSDS, {B7C62014-02BD-4F35-9718-104CE8CFB14C}, 1, c:\LocalBuilds\1\2\Sources\Source\Calculator\Calculator\obj\Debug\Calculator.pdb

You can see the GUID highlighted above. If you were to go check out the Symbol Server file share, you can also find the GUID used to differentiate between all of the different versions of the symbol files that are stored for a particular assembly.

Update:  (2/15/2011) I learned something new from Chris Schmich from the Visual Studio Diagnostics team.  He indicated that the PDB age (which is highlighted above in green) is also used to match the symbols.  You’ll notice that the PDB age for all of my symbols is 1 and is appended to the end of the GUID when stored in Symbol Server.  Thanks Chris for the extra information!

image

IntelliTrace Files and Symbol Server

I also wanted to mention that when testers use Microsoft Test Manager and run manual test cases where they have collected IntelliTrace logs, you’ll notice that when you open one of those IntelliTrace logs (for example attached to a bug work item) you will see the Symbol Server location that was collected from the assembly being tested as well:

image

This green-light should be awesome for you as a developer now because you can connect to the Symbol Server location and start debugging using the IntelliTrace log and the Source Server information contained inside of the symbols.

Retention Policies

One other thing to consider: as you have more and more builds performed using TFS 2010 Build, you’ll want to set up your retention policies. The Symbol Server file share can start to go up in size pretty quickly so you can have the retention policies also delete the corresponding symbols from Symbol Server if you choose by setting the “What to Delete” option.

image

SNAGHTML25e109f

You want to also make sure, however, that any “Released” builds should be marked as “Retain Indefinitely” to ensure that the retention policies never delete the symbols (or anything else about the build for that matter!)

image

Summary

There you go! Your developers will be very appreciative whenever all of this is setup. You’ll have a system that stores your symbols for whenever you need them and those symbols will have information to let the debugging utilities know where to grab the original source code from the TFS version control repository.

Ed Blankenship



Tuesday, March 22, 2011 8:42:59 AM (Pacific Standard Time, UTC-08:00)
Hi,

Can you use this same approach with C++ projects in TFS 2010? Were having a problem getting the symbol files to copy over to the network share we setup. We've changed the settings for Index Sources to True, and set the Path to Publish Symbols to our share folder.

Any ideas or suggestions would be appreciated!

Larry B.
Larry B
Tuesday, March 22, 2011 1:04:22 PM (Pacific Standard Time, UTC-08:00)
Hi Larry,

You should absolutley be able to use this same approach with unmanaged executables and libraries. There is slightly different information stored in the symbols for unmanaged projects but the same concepts apply. You mentioned you set the build process parameters to True but are you having any specific issues with the source indexing and symbol publishing process? Let me know and we can see if we can track those down.

Take care,

Ed B.
Thursday, August 04, 2011 8:13:46 AM (Pacific Daylight Time, UTC-07:00)
This is not always working fine for me and I didn't figure out exactly yet why.

If I add a reference onto assemblies located in the TFS Build's drop folder (i.e.: not deployed in the GAC), then I can debug the code without any issue; both the related .pdb and sources are found by the debugger.

But when I reference assemblies in the GAC, I don't get the source code. The debugger is trying to locate the source in a path which only exist on the Build Server. So, the "source indexing" seems to be missing. However, looking into the build log, it was clearly well executed.

Could this be due to the fact that I signed the assemblies after the "source indexing" ? (The assemblies are only delay signed during the build). Or could it be due to the fact that the assemblies are packaged into an MSI (created with an old "VS setup project") which deploys them into the GAC ?

V.
Valery
Tuesday, January 10, 2012 5:51:07 PM (Pacific Standard Time, UTC-08:00)
Hi Valery!

I think you may be right. I'm not quite sure if you sign assemblies after compilation whether that updates it in a way where the symbols won't match any longer. Are you able to modify your build process so that the signing is done during compilation and before the phase where source indexing and symbol publishing occurs?
Monday, January 23, 2012 5:38:42 AM (Pacific Standard Time, UTC-08:00)
Ed, do you know of any way to publish the sources with the symbols? We don't want everyone who ever needs to step through these sources to have to have access to these TFS projects.

Tuesday, January 24, 2012 3:53:36 AM (Pacific Standard Time, UTC-08:00)

Ed, do you know of any way to publish the sources with the symbols?


That's a great question that I'll need to look into a little more. I have researched into the public source server approach and you can definitely do it but the needed infrastructure will need to be manually built out. Feel free to contact me on my blog site (using the Contact link) if you want to talk more about it.

P.S. Looks like I'll need to look into what's up with the dasBlog Open ID functionality with Google accounts because I have no idea of what your e-mail address, web site link, or name details actually are. :)
Monday, March 19, 2012 5:34:41 PM (Pacific Standard Time, UTC-08:00)
Hi,

I have done successfully setup Symbol server in TFS. But i need to exclude third part pdb file at that time of publish to Symbol server.
The optiones available in client side, but i need to setup in server side (TFS build).

Can you help me to resolve this problem.

Advance thanks
Rajesh
Rajesh
Tuesday, March 20, 2012 5:39:48 AM (Pacific Standard Time, UTC-08:00)
I have done successfully setup Symbol server in TFS. But i need to exclude third part pdb file at that time of publish to Symbol server.
The options available in client side, but i need to setup in server side (TFS build).


Hi Rajesh,

I don't of a particularly easy way to handle this scenario off the top of my head. It would involve customizing the build process template and removing the specific .PDB entries that you don't want to be published from the FileList collection that is passed into the Publish Symbols build workflow activity.
Thursday, November 08, 2012 3:51:57 AM (Pacific Standard Time, UTC-08:00)
Hi Ed,
I want to enable source code debugging for clients that use our framework. I followed steps described in this article and unfortunately get stuck in the last step. When I press step into our framework class method the security window appears with this command: tf.exe view /version:21 /noprompt "$/Framework/Core/Core/Class1.cs" /server:http://my-pc:8080/tfs/FrameworkCollection /console >"C:\DOCUME~1\PRESEN~1\LOCALS~1\APPLIC~1\SOURCE~1\VSTFSSERVER\Framework\Core\Core\Class1.cs\Class1;C21.cs".
I assume that the command retrieves appropriate source file version and want to put it on C:\DOCUME~1\PRESEN~1\LOCALS~1\APPLIC~1\SOURCE~1\VSTFSSERVER\Framework\Core\Core\Class1.cs\Class1;C21.cs. Actually it does not work. When I try to execute the command via VS command prompt and error appears:
"TF30063: You are not authorized to access my-pc\FrameworkCollecion".
Do You have any idea how to fix it?

Thanks for the great article and I'm looking forward to hearing from You
Steve
Thursday, November 08, 2012 5:07:19 AM (Pacific Standard Time, UTC-08:00)
Hi Ed,
I know where the problem lies: The command from above(tf.exe view /version:21 /noprompt "$/Framework/Core/Core/Class1.cs" /server:http://my-pc:8080/tfs/FrameworkCollection /console >"C:\DOCUME~1\PRESEN~1\LOCALS~1\APPLIC~1\SOURCE~1\VSTFSSERVER\Framework\Core\Core\Class1.cs\Class1;C21.cs")
is missing the /login section. When I added proper credential (and fired it manually via vs command ine)the file was obtained and everything works fine ( I could step into framework code).
Now the question is why the window for providing user credential does not appear when stepping into framework code? How to force it to ask for the credential ?
Thanks!
Steve
Thursday, February 28, 2013 1:16:33 PM (Pacific Standard Time, UTC-08:00)
Ed,
Thanks for the article. Are there any significant updates in VS / TFS 2012 tht impact how this should be approached? Any movement on incorporating Symbol and / or Source server capabilities directly into TFS?

Thx,
JF
Jay Frysinger
Wednesday, April 02, 2014 6:42:58 PM (Pacific Standard Time, UTC-08:00)
Hello, Ed,

I read this article some time ago, and indexing symbols showed itself be something very useful. But sometimes the resources for setting up a build server are not available. So I figured out how it works, and started making some script to index my symbols.

Recently I came up with a very useful NuGet package, that you just install it in your project, and it takes care about indexing symbol files after build. If you or someone would like to check it you, it's under the name TfsIndexer on NuGet's official repository (and also CodePlex).

Thanks for showing me the way on how to take good care of my symbol files.

Eduardo Sousa
Wednesday, May 07, 2014 6:13:33 AM (Pacific Daylight Time, UTC-07:00)
Hello,
We are configured with a non default build template: we use Upgrade Template.
Is there still a way to easily setup a Symbol Server?
Thanks!
Shlomo Am Shalem
Tuesday, August 12, 2014 12:15:38 PM (Pacific Daylight Time, UTC-07:00)
Hello Ed, thanks for all the great articles.

I’ve read this article as well as the ones you referenced and I’d like to confirm something. I want to turn off indexing and I want to be sure I understand what I’ll lose.

If I don’t index the source during a build but generate and deliver the PDB files with my application, I won’t be able to use a debugger on the application but I will still get stack traces and line numbers when exceptions are thrown/logged, right? I could live with that and do a special build if/when remote debugging is needed.

Why do this? Long story short, source indexing it taking too long in our environment, our Build and TFS servers are remote from each other and something at the network level changed a few weeks ago (who knows what). Indexing went from 1 hour to several, I reduced logging to minimal and managed to index in 4 hours but that’s still too long. Seems that source indexing generates a lot of log messages from the build to the TFS server so it can track the status.

Thanks,
Martin.
Martin
Name
E-mail
(will show your gravatar icon)
Home page

Comment (Some html is allowed: a@href@title, b, blockquote@cite, em, i, strike, strong, sub, sup, u) where the @ means "attribute." For example, you can use <a href="" title=""> or <blockquote cite="Scott">.  

[Captcha]Enter the code shown (prevents robots):

Live Comment Preview