Note: This entry has been sitting for a long time … I’ve been very, very, very busy managing the builds for over 100 projects. There is more to come as I share the learnings (and code) from this.

I’m aware that this is for the now “legacy” version of TFS. But as we’ve moved our implementation from 2008 to 2010, there are still a couple of things that I’ve done there that, I think, are really cool that I’d like to share before they fade away into the oblivion of previous versions. This is the first of these.

First, some background. We have a pretty well defined and structured deployment process at work. It’s been in place for quite some time and wasn’t anything that was going to change when I came in. Besides that, it made a good deal of sense. Here’s the process as it was when I arrived:

1. Development team creates a deployment package. This package includes both the source and the compiled/release files in separate zip files.
2. For some deployments, there would also be an associated MSI file.
3. A deployment would go through a “dev” environment, then a “test” environment before being (perhaps) approved for production release. The development team would create an item in a Sharepoint list for the deployment. They created one of these for both dev and test environment deployments. Deployment versions were based in the ID of this item. More on this in a future post.
4. The deployment manager kept a source safe database of all deployments, as well as copies on a share.

That covers the high points. Note that there were several items common to all deployments – zips for both source and release, ID’s based on a separate tracking item and potentially MSI files. I also implemented a branching strategy based on how projects worked on the team. Again, more on this item in a future post. But the main thing that I want to point out is that there are a bunch of things that are common to each and every build – and things that can be automated. And that’s what I’ll be talking about.

Since TFS Build 2008 is based on MSBuild files, that was where we needed to work. The first step was to download the MSBuild Extension Pack – all of the tasks are useful and some, in particular, were highly applicable to our process. So that did need to be installed on the build server. I certainly didn’t want to do a copy-paste into all of the project build templates (which will eventually be over 100); that wouldn’t allow me to make any changes to all builds and all projects in an efficient, sane manner. Starting with a pre-built template wouldn’t be any better either as I’d have the same problem. But … MSBuild does have a facility to share tasks and targets among multiple MSBuild project files, one that every Visual Studio project (that supports MSBuild – not all do!) uses. This is through the Import element. It’s reminiscent of the “Magical Mystery Includes” from ASP Classic days but that doesn’t negate it’s usefulness. You just need to make sure that you document it!

And that, my friends, is the key to making this work. I created a MSBuild project file that handles all of the common items. It created a custom version number based on the deployment request and the date. It created the zip files for source and release and then copied them to the drop folder. It would also handle calling Visual Studio from the command line and building the MSI file. It also imported the MSBuild Extension Pack tasks. All of these tasks were done in documented extensibility targets for TFS Build. This single MSBuild project file would then be included into every individual build project file; it would do its thing and the project file could also then do a couple of items unique to the project. Tasks that were common, but not in every project (like building the MSI) were controlled by properties that could be set either via the command line or in the project’s build file. The ID of the deployment request was also a property included in the MSBuild command line; this was used to create the version number for the deployment, enabling better auditing and trackability. There is also a command-line switch that runs the build in a “Test Mode” that doesn’t label and creates different build numbers.

But that’s not all. Most of what I described above is provide either out-of-the-box or with tasks from the MSBuild Extension Pack. But … I mentioned the branching strategy and that is a very important part of the entire process. We have both support and development teams working on the same project (at times). They need to work in parallel and to be able to merge their changes together. We have also had cases where both the dev team and the support team were looking to deploy (eventually) to production within a week of each other so we needed to be able to integrate the changes for a single package. I won’t get into the details of that but I wanted to mention it because it has a significant impact on build … we need to be able to build, using the exact same build script, regardless of the source branch. And that is where I found nothing that could help me. So what did I do? I wrote code, of course!

So I wrote a task that changes the workspace mappings based on MSBuild properties – supplied via MSBuild arguments – before the build workspace is created and any files are retrieved from source control. This allows a build to be done from any branch from a single build script … which I like.

At the end of all of this … some of the process described about was implemented completely via MSBuild for TF Build 2008, but not all. I stopped working on further enhancements when I got the go-ahead for migrating to TFS 2010 shortly after RTM - we upgrading within a week of RTM. I’ve been working on a lot of stuff since then, including (but not limited to) build activities that make my life easier. I’ll be sharing some of that in the coming weeks/months, so check back.

I went to the “Next Generation Testing with Visual Studio 2010” event in Houston yesterday. And … while it is somewhat strange going to a Microsoft event where I’m a “mere attendee” … I did really enjoy the experience. There was a TON of information there and, for me, one of the very coolest technology revolved around lab management and the automation of Hyper-V based environments to do this. Abel (I hope that I spelled his name correctly) talked very deeply and extensively about his experience working with Lab Manager; very good stuff as he talked about real-world challenges around this. Plus he’s a biker … he and I chatted for quite some time after the event about motorcycles and motorcycling and that, in itself, is worth 20 points (out of 100) in my book.

One thing that he mentioned was the cost of copying VHD’s. There is only so much that can be done about that but … it struck me … as he was talking about it, it seemed that he was using his test servers as AD controllers. Maybe not … but the expense of AD controllers, etc. … was mentioned. And yes, little things like AD controllers (and DNS and DHCP) are very important when creating a dev/test environment. And then … and this may be just me … I like to keep dev/test environments as isolated as possible from production environments. A dev/test environment is meant to be blown up. It should be a place where crazy what-if scenarios (as well as normal testing) can be tried and evaluated without any fear of disrupting real work.  This includes AD domains … the test environment should have its own, dedicated AD. If there is a trust, then it’s a one-way trust and all traffic goes through a firewall between test and production environments. For Lab Management to work effectively and smoothly, you’ll need to do this.

This type of environment is hard to set up. TFS 2010 certainly makes the test virtual servers easy to set up and configure but it doesn’t seem to help with the core networking services that need to be there. But then, if I was a PM on that team, I’d put that task well “out of scope”. Probably forever … putting that “in scope” would require dealing with a core network environment that is completely unpredictable. That’s a recipe for Ugly.

But … configure the core network services and it’s all easier. Core network services include Active Directory (authT and authZ), DNS (name resolution), DHCP (IP Address independence) and RRAS (routing outside the test network). These are core network services that developers just expect to “work” – as well they should – but most developers never actually work with them. I learned this stuff “way back in the day” when you needed to understand the details of the underlying specs (can you say IUnknown?) to debug stuff. Fortunately, development has moved beyond the point where that is important on a daily basis. So … some details …

Windows Server 2008+ Server Core

It doesn’t matter if it’s Win2K8 or 2K8 R2. Server Core is your friend for any and all test environments. Create a virtual machine with Server Core and you’ve got AD, DNS and DHCP services for your test environment at a pretty low (virtual machine) cost. I run my server core VM at 256MB … I’ve tried less but it didn’t work well; 256MB seems to be the (practical) lower limit.

On this Server Core VM, add Active Directory Services, DNS Services and DHCP Services. It’s all command-line stuff that’s pretty well documented on TechNet. Connect to the Server Core instance with remote administration tools and set your DNS and DHCP environments up. Dealing with IP settings on individual machines with static addresses is far too painful and time-consuming for an effective test environment. Plus, I can never remember what IP is where anyway. Then, set this little Server Core VM to run all the time. Set all of your virtual machines to use your internal network with DHCP enabled. If you do/want a static address, use DHCP to create a reservation that assigns the same IP address to the same NIC based on the physical (MAC) address.

SysPrep

For your virtual machines, Sysprep is your friend. Setting up the core OS can be a pain, especially when the same roles and role services are needed over and over and over and over again. Wouldn’t it be nice to just do all of this common stuff once? Sysprep lets you do that … set up your server roles (Application Server, DNS, DHCP, IIS …) and then run sysprep (C:\%System32%\Sysprep\sysprep.exe). Select “Enter System Out-of-Box Experience” with “Generalize” checked. Save the VHD and use it as the base VHD for your test servers. Not differencing disks but a copy-rename-attach process. Then use your “sysprep’d” VHD to start your test virtual machines. They will go through a mini-setup and then you’ll have a running server that’s ready for its specific functionality in just a couple of minutes. Tip: when you are creating your sysprep’d image, make sure that you apply all of the updates before sysprepping. It won’t make anything break, but it’s helpful to have all of them already applied. And … you can also install some software before sysprepping. Antivirus, Visual Studio and Office are my typical candidates; I do try to avoid server products (i.e Sql Server, MOSS, etc.) in the sysprepped image. Be prepared to rev your base Sysprep’d image from time to time, usually with just new updates.

RRAS

RRAS stands for “Routing and Remote Access Services”. In Windows Server 2008, this is under “Network Policy and Access Services”. This allows you to bridge a private virtual network with the public Internet. It also allows you to route between two different isolated test networks. Finally, you can use it to allow access to wireless networks from a Hyper-V virtual network. Install NPAS on the host machine and set it up as either a regular router for a complex, enterprise implementation with domain trusts and the like or as a NAT router just to get Internet access for your virtual machines. Just to repeat … you can also use this to allow Hyper-V virtual networks to use a wireless connection for connecting to the Internet.

All of this … I won’t say it’s simple. But it’s really not all that hard, either … there are a lot of “dotting i’s and crossing t’s”. It helps – tremendously – if you know the basics of how networking, TCP/IP and routing work. You do not need to know this in the depth required to implement a 30,000+ desktop Active Directory implementation; that’s a whole different ballgame.

Microsoft has two project management products … Microsoft Project (and the accompanying Project Server) and Team Foundation Server. Project is much more general; it’s designed to manage any type of project. TFS is very specialized; it’s designed to manage software projects. So one could reasonably say that TFS, from a project management perspective, focuses on a specific use case for Project. TFS’s Project integration certainly helps to make that case.

Both TFS and Project Server integrate with Sharepoint for collaboration. That makes sense; there is no reason for Microsoft to reinvent the wheel when it comes to web-based collaboration. Sharepoint does that and it does that pretty well. Extending Sharepoint for this additional functionality is certainly the best use of limited development resources.

But what happens when an organization uses both Project Server and TFS? The question – and the debate – over which implementation to use for storing project-related documents in inevitable. Both add features to the Sharepoint site – TFS adds “Team Collaboration Lists” and Project Server adds “Project Workspace Collaboration Lists”. I really don’t see any clear, undisputable reason to choose one over the other. The path of least resistance is to have one site for TFS and one for Project Server; but this leads to confusion for the project team – what document goes where? Or … you have a project manager that gets addicted to the drag-drop upload providing by the Team Foundation Client when the organization has chosen to put all essential deliverable documents in the Project Server Project Workspace. Or … you have a project manager that works exclusively in Project Server and gets … annoyed … that the developers have to go to another site to find the docs. TFS was supposed to solve this and it does – if you only use TFS.

Regardless, it all violates the Prime Directive of software development – the KISS principle. That’s "Keep It Simple, Stupid” for those of you that haven’t heard it. Developing world-class enterprise software is complicated enough as it is. There is absolutely, positively no need to introduce additional layers of complexity that have no direct business value to the project. Simplicity is a virtue that I wish more developers had – and one of the reasons that I love TFS is that, when used well, it helps simplify the tedious aspects of software projects (I’ve always hated status reports!) in a way that is easy for a developer to do while working on the project. That’s the tip of the iceberg but I’ll stop there.

All that babbling aside, I’m sure that you, dear reader, have figured out by now that Project Server and TFS are both being used by the organization that I’m currently working with to implement TFS. And I was issued the challenge to bring these two products together to at least use the same Sharepoint site for their workspaces. I found this article but I thought that there must be a simpler way to do it. And yes, there is. The method that you use depends on whether you create the TFS project (and associated Sharepoint site) or the Project Server workspace first. I’ll detail both.

Requirements for Both Solutions

• The Project Server Sharepoint components need to be installed on the target Sharepoint server.
• The Team Foundation Server Sharepoint components need to be installed on the target Sharepoint server.
• Use TFSAdminUtil ConfigureConnections to point the TFS server to the target Sharepoint server and related URIs. All TFS-related Sharepoint sites will use this server and top-level URI. If you have existing Sharepoint/TFS sites, you will need to migrate them to the new server.
• The TFS administrator will need administrator permissions on Project Server.
• You will need to use TFSConfigWss to configure the URLs for Sharepoint on the target server.
• The TFS top-level URL needs to be a wildcard inclusion. An explicit inclusion won’t work. Well, maybe it will, but I don’t know how to make it work. Maybe someone with deeper Sharepoint knowledge will show how to do this. I certainly hope so.
• Project Server does not need to default to the same site collection as TFS. Project Server’s Sharepoint integration is more flexible than TFS’s.

Project Workspace First

• Customize your TFS project template: Remove references to the Sharepoint task in . Since the site already exists, you will get an error creating the project unless you remove the Sharepoint tasks from project creation.
• Remove (comment) references to the Sharepoint plugin (under the plugins element).

<plugin name="Microsoft.ProjectCreationWizard.Portal" wizardPage="true" />

• Remove the portal groups

<group id="Portal" description="Creating project Site" completionMessage="Project site created.">
  <dependencies>
    <dependency groupId="Classification" />
    <dependency groupId="WorkItemTracking" />
    <dependency groupId="VersionControl" />
  </dependencies>
  <taskList filename="Windows SharePoint Services\WssTasks.xml" />
</group>



• Create your project workspace. Make sure that it uses the same top-level URL that TFS uses. Fortunately, Project Server lets you do this.
• Create the Team Project using the template that excludes Sharepoint and with the same name as the Project Server workspace. The naming is important; if the names don’t match for the Project Workspace and the Team Project, it won’t work. Fortunately, Project Server doesn’t tie you to the project name for the workspace.
• Downside: The TFS template won’t add any documents, document libraries or reports. Since that performs some “magic” to Excel/Project lists that are tied to the Team Project, you lose that. And you’ll also have to add any reports to the project manually.

Team Foundation Project First

• Setup:
1. Don’t require project workspace creation when a new project is published (PWA … Server Settings … Project Workspace Provisioning Settings – Set “Allow users to manually create project workspaces in Project Server and Save).
2. Create a blank Team Project based on your template. Activate the “Project Workspace Collaboration Lists” feature for the site (that’s the only additional feature that you need).
3. Create a template from the site but don’t include content – the TFS template will add this. .
4. Specify your new template as the template for the TFS project creation wizard (Windows SharePoint Services\WssTasks.xml). The template should be global on the Sharepoint server.
   

<Portal>
        <site template="TFS-Project" language="1033" />
        ...
</Portal>



• Customize your TFS Project: Specify the target template for the TFS task to your new template. Since the Sharepoint project template (STP) already has the Project Server features activated, you don’t need to do that step. During project creation, TFS will add all of your default document libraries and documents. This includes whatever majik they do to make Excel and Project files in the template work with the proper Team Project.
• Publish the Project: but … don’t include the Project Workspace. See link referenced above. You will not get the option for "Do not create a workspace at this time” unless you do Setup step 1 (a step that Neil didn’t mention).
• Associate the Project Workspace with the TFS site: In the article referenced above, Neil details this quite well. I won’t repeat it here.
• Downside: Require manual association of the project workspace Sharepoint site with the TFS Sharepoint site.

Neither option uses undocumented extensibility points for TFS 2008. And yes, there are variations between the two that work – with unique sets of good/bad/ugly. This post is long enough without going into all of the possible permutations.

Very Important Disclaimer: While nothing here falls outside of the documented extensibility points for TFS 2008, I doubt that anything is in the product team’s upgrade test cases – especially since nothing here seems to be documented anywhere else.  I have not yet tested the upgrade path for either of these techniques to TFS 2010. Considering the (very frustrating) lack of guidance and documentation for these techniques, I cannot say anything one way or the other. I will, however, be testing and experimenting with the upgrade paths (Hyper-V shapshots ROCK!!!) and any goodness that I discover will be published here. Of course, I’m more than happy to work with the TFS product team on this …

There comes a time in every TFS implementation where you need to create your own set of default project work items in your template. You can do this straight in the XML by editing workitems.xml in your process template’s Work Item Tracking folder. I said you could, not that you’d actually want to! You can also use the Process Editor to add them one at a time like you do with the Team Foundation client. Uggh. Wouldn’t it be better just to use Excel to put them into a scratch, temporary TF Project? I certainly think so; I can enter work items in Excel a lot faster than I can in the TF Client (or in the Process Editor).

I did some looking about thinking that someone must have solved this already and posted some code, open source, whatever. There was one result that looked promising but … the URL didn’t work (so I’m not posting it here). <Heavy Sigh>

Oh well, time to pop open Visual Studio and start coding …

It turned out that it was pretty easy, in fact. In the sample below, you take a raw work item query (in WIQL) and get the list of work items that you want to export with the columns that you also want to export. The order doesn’t matter. From there, you execute the query against the Work Item Store to get the list of work items. Loop over them and write to XML in the format that the project template uses.

It is, for the most part, really that simple. There are only 2 things that you really have to watch out for. First, the project name should be replaced with a marker for the target project name  ($$PROJECTNAME$$). Second, you need to make sure that the field is applicable to the work item type. This is something that you need to be especially aware of if you have multiple work item types in your default work item set. With all of that … I am proud to present to you … the actual code (exception handling removed for clarity … put yours in!)

public void Export(
    string serverName, string projectName, string filePath, string WIQL)
{
    TeamFoundationServer tfs = TeamFoundationServerFactory.GetServer(
                            serverName, 
                            new UICredentialsProvider() );
    tfs.Authenticate();
    WorkItemStore store = 
        (WorkItemStore)tfs.GetService(typeof(WorkItemStore));
    WorkItemCollection workItems = store.Query(WIQL);
    using (System.IO.Stream output = 
        System.IO.File.Open(filePath, 
            System.IO.FileMode.Create, 
            System.IO.FileAccess.ReadWrite))
    {
        using (XmlWriter writer = XmlWriter.Create(output))
        {
            writer.WriteStartDocument(true);
            writer.WriteStartElement("WORKITEMS");
            WriteWorkItems(projectName, workItems, writer);
            writer.Flush();
            writer.Close();
        }
        
    }
}

private static void WriteWorkItems(string projectName, 
    WorkItemCollection workItems, XmlWriter writer)
{
    for (int i = 0; i < workItems.Count; i++)
    {
        var workItem = workItems[0];
        writer.WriteStartElement("WI");
        writer.WriteAttributeString("type", workItem.Type.Name);
        foreach (FieldDefinition field in workItems.DisplayFields)
        {
            if (workItem.Fields.Contains(field.Name))
            {
                writer.WriteStartElement("FIELD");
                writer.WriteAttributeString("refname", field.ReferenceName);
                writer.WriteAttributeString("value", 
                    workItem.Fields[field.Name].Value.ToString().Replace(
                            projectName, "$$PROJECTNAME$$"));
                writer.WriteEndElement();
            }
        }
        writer.WriteEndElement();
    }
}

Now, I don’t know about you, but when I see “God Mode”, I think of running around in some shooter killing all the bad guys/monsters/aliens/whatever without any worry of damage or harm coming to me. Those of you that cut your teeth on some of the old school DOS-based games of yester-century (a la Doom) will know exactly what I’m talking about.

I don’t think of “God Mode” when it comes to an operating system. Unless, perhaps, running everything as admin. Really, though, that’s probably more akin to anti-God Mode if you think about it; you need to be a little more worried about potential damage and hazards since you are running as admin.

But then there’s Windows 7 and we have things like User Account Control to keep you from doing too much harm. So is “God Mode” some super-anti-UAC thingie that makes it like it was in the old days when you could call deltree from the root of your C: drive and it would happily go about doing it? (Note: Yes, I know someone that did this. No, it wasn’t me. And no, I don’t recommend trying it.)

Nope, not at all like the God Mode that I knew and loved from the days of Doom.

So when I saw an article called “Understanding Windows 7's 'GodMode'”, I was intrigued. It’s not the “God Mode” of Doom-yore, but it is interesting nonetheless and I’m kinda liking it. You create a folder and name it “GodMode.{ED7BA470-8E54-465E-825C-99712043E01C}” (no quotes, of course). Then … behold! the icon changes and you have access to a ton of controls and settings for the operating system. It does work on Windows Server 2008 R2 x64, which is what I’m running right now.

It reminds me, in a way, of the Windows 95 Product Team Easter Egg. But simpler to create and a lot more useful.