I went to the “Next Generation Testing with Visual Studio 2010” event in Houston yesterday. And … while it is somewhat strange going to a Microsoft event where I’m a “mere attendee” … I did really enjoy the experience. There was a TON of information there and, for me, one of the very coolest technology revolved around lab management and the automation of Hyper-V based environments to do this. Abel (I hope that I spelled his name correctly) talked very deeply and extensively about his experience working with Lab Manager; very good stuff as he talked about real-world challenges around this. Plus he’s a biker … he and I chatted for quite some time after the event about motorcycles and motorcycling and that, in itself, is worth 20 points (out of 100) in my book.

One thing that he mentioned was the cost of copying VHD’s. There is only so much that can be done about that but … it struck me … as he was talking about it, it seemed that he was using his test servers as AD controllers. Maybe not … but the expense of AD controllers, etc. … was mentioned. And yes, little things like AD controllers (and DNS and DHCP) are very important when creating a dev/test environment. And then … and this may be just me … I like to keep dev/test environments as isolated as possible from production environments. A dev/test environment is meant to be blown up. It should be a place where crazy what-if scenarios (as well as normal testing) can be tried and evaluated without any fear of disrupting real work.  This includes AD domains … the test environment should have its own, dedicated AD. If there is a trust, then it’s a one-way trust and all traffic goes through a firewall between test and production environments. For Lab Management to work effectively and smoothly, you’ll need to do this.

This type of environment is hard to set up. TFS 2010 certainly makes the test virtual servers easy to set up and configure but it doesn’t seem to help with the core networking services that need to be there. But then, if I was a PM on that team, I’d put that task well “out of scope”. Probably forever … putting that “in scope” would require dealing with a core network environment that is completely unpredictable. That’s a recipe for Ugly.

But … configure the core network services and it’s all easier. Core network services include Active Directory (authT and authZ), DNS (name resolution), DHCP (IP Address independence) and RRAS (routing outside the test network). These are core network services that developers just expect to “work” – as well they should – but most developers never actually work with them. I learned this stuff “way back in the day” when you needed to understand the details of the underlying specs (can you say IUnknown?) to debug stuff. Fortunately, development has moved beyond the point where that is important on a daily basis. So … some details …

Windows Server 2008+ Server Core

It doesn’t matter if it’s Win2K8 or 2K8 R2. Server Core is your friend for any and all test environments. Create a virtual machine with Server Core and you’ve got AD, DNS and DHCP services for your test environment at a pretty low (virtual machine) cost. I run my server core VM at 256MB … I’ve tried less but it didn’t work well; 256MB seems to be the (practical) lower limit.

On this Server Core VM, add Active Directory Services, DNS Services and DHCP Services. It’s all command-line stuff that’s pretty well documented on TechNet. Connect to the Server Core instance with remote administration tools and set your DNS and DHCP environments up. Dealing with IP settings on individual machines with static addresses is far too painful and time-consuming for an effective test environment. Plus, I can never remember what IP is where anyway. Then, set this little Server Core VM to run all the time. Set all of your virtual machines to use your internal network with DHCP enabled. If you do/want a static address, use DHCP to create a reservation that assigns the same IP address to the same NIC based on the physical (MAC) address.

SysPrep

For your virtual machines, Sysprep is your friend. Setting up the core OS can be a pain, especially when the same roles and role services are needed over and over and over and over again. Wouldn’t it be nice to just do all of this common stuff once? Sysprep lets you do that … set up your server roles (Application Server, DNS, DHCP, IIS …) and then run sysprep (C:\%System32%\Sysprep\sysprep.exe). Select “Enter System Out-of-Box Experience” with “Generalize” checked. Save the VHD and use it as the base VHD for your test servers. Not differencing disks but a copy-rename-attach process. Then use your “sysprep’d” VHD to start your test virtual machines. They will go through a mini-setup and then you’ll have a running server that’s ready for its specific functionality in just a couple of minutes. Tip: when you are creating your sysprep’d image, make sure that you apply all of the updates before sysprepping. It won’t make anything break, but it’s helpful to have all of them already applied. And … you can also install some software before sysprepping. Antivirus, Visual Studio and Office are my typical candidates; I do try to avoid server products (i.e Sql Server, MOSS, etc.) in the sysprepped image. Be prepared to rev your base Sysprep’d image from time to time, usually with just new updates.

RRAS

RRAS stands for “Routing and Remote Access Services”. In Windows Server 2008, this is under “Network Policy and Access Services”. This allows you to bridge a private virtual network with the public Internet. It also allows you to route between two different isolated test networks. Finally, you can use it to allow access to wireless networks from a Hyper-V virtual network. Install NPAS on the host machine and set it up as either a regular router for a complex, enterprise implementation with domain trusts and the like or as a NAT router just to get Internet access for your virtual machines. Just to repeat … you can also use this to allow Hyper-V virtual networks to use a wireless connection for connecting to the Internet.

All of this … I won’t say it’s simple. But it’s really not all that hard, either … there are a lot of “dotting i’s and crossing t’s”. It helps – tremendously – if you know the basics of how networking, TCP/IP and routing work. You do not need to know this in the depth required to implement a 30,000+ desktop Active Directory implementation; that’s a whole different ballgame.

Microsoft has two project management products … Microsoft Project (and the accompanying Project Server) and Team Foundation Server. Project is much more general; it’s designed to manage any type of project. TFS is very specialized; it’s designed to manage software projects. So one could reasonably say that TFS, from a project management perspective, focuses on a specific use case for Project. TFS’s Project integration certainly helps to make that case.

Both TFS and Project Server integrate with Sharepoint for collaboration. That makes sense; there is no reason for Microsoft to reinvent the wheel when it comes to web-based collaboration. Sharepoint does that and it does that pretty well. Extending Sharepoint for this additional functionality is certainly the best use of limited development resources.

But what happens when an organization uses both Project Server and TFS? The question – and the debate – over which implementation to use for storing project-related documents in inevitable. Both add features to the Sharepoint site – TFS adds “Team Collaboration Lists” and Project Server adds “Project Workspace Collaboration Lists”. I really don’t see any clear, undisputable reason to choose one over the other. The path of least resistance is to have one site for TFS and one for Project Server; but this leads to confusion for the project team – what document goes where? Or … you have a project manager that gets addicted to the drag-drop upload providing by the Team Foundation Client when the organization has chosen to put all essential deliverable documents in the Project Server Project Workspace. Or … you have a project manager that works exclusively in Project Server and gets … annoyed … that the developers have to go to another site to find the docs. TFS was supposed to solve this and it does – if you only use TFS.

Regardless, it all violates the Prime Directive of software development – the KISS principle. That’s "Keep It Simple, Stupid” for those of you that haven’t heard it. Developing world-class enterprise software is complicated enough as it is. There is absolutely, positively no need to introduce additional layers of complexity that have no direct business value to the project. Simplicity is a virtue that I wish more developers had – and one of the reasons that I love TFS is that, when used well, it helps simplify the tedious aspects of software projects (I’ve always hated status reports!) in a way that is easy for a developer to do while working on the project. That’s the tip of the iceberg but I’ll stop there.

All that babbling aside, I’m sure that you, dear reader, have figured out by now that Project Server and TFS are both being used by the organization that I’m currently working with to implement TFS. And I was issued the challenge to bring these two products together to at least use the same Sharepoint site for their workspaces. I found this article but I thought that there must be a simpler way to do it. And yes, there is. The method that you use depends on whether you create the TFS project (and associated Sharepoint site) or the Project Server workspace first. I’ll detail both.

Requirements for Both Solutions

• The Project Server Sharepoint components need to be installed on the target Sharepoint server.
• The Team Foundation Server Sharepoint components need to be installed on the target Sharepoint server.
• Use TFSAdminUtil ConfigureConnections to point the TFS server to the target Sharepoint server and related URIs. All TFS-related Sharepoint sites will use this server and top-level URI. If you have existing Sharepoint/TFS sites, you will need to migrate them to the new server.
• The TFS administrator will need administrator permissions on Project Server.
• You will need to use TFSConfigWss to configure the URLs for Sharepoint on the target server.
• The TFS top-level URL needs to be a wildcard inclusion. An explicit inclusion won’t work. Well, maybe it will, but I don’t know how to make it work. Maybe someone with deeper Sharepoint knowledge will show how to do this. I certainly hope so.
• Project Server does not need to default to the same site collection as TFS. Project Server’s Sharepoint integration is more flexible than TFS’s.

Project Workspace First

• Customize your TFS project template: Remove references to the Sharepoint task in . Since the site already exists, you will get an error creating the project unless you remove the Sharepoint tasks from project creation.
• Remove (comment) references to the Sharepoint plugin (under the plugins element).

<plugin name="Microsoft.ProjectCreationWizard.Portal" wizardPage="true" />

• Remove the portal groups

<group id="Portal" description="Creating project Site" completionMessage="Project site created.">
  <dependencies>
    <dependency groupId="Classification" />
    <dependency groupId="WorkItemTracking" />
    <dependency groupId="VersionControl" />
  </dependencies>
  <taskList filename="Windows SharePoint Services\WssTasks.xml" />
</group>



• Create your project workspace. Make sure that it uses the same top-level URL that TFS uses. Fortunately, Project Server lets you do this.
• Create the Team Project using the template that excludes Sharepoint and with the same name as the Project Server workspace. The naming is important; if the names don’t match for the Project Workspace and the Team Project, it won’t work. Fortunately, Project Server doesn’t tie you to the project name for the workspace.
• Downside: The TFS template won’t add any documents, document libraries or reports. Since that performs some “magic” to Excel/Project lists that are tied to the Team Project, you lose that. And you’ll also have to add any reports to the project manually.

Team Foundation Project First

• Setup:
1. Don’t require project workspace creation when a new project is published (PWA … Server Settings … Project Workspace Provisioning Settings – Set “Allow users to manually create project workspaces in Project Server and Save).
2. Create a blank Team Project based on your template. Activate the “Project Workspace Collaboration Lists” feature for the site (that’s the only additional feature that you need).
3. Create a template from the site but don’t include content – the TFS template will add this. .
4. Specify your new template as the template for the TFS project creation wizard (Windows SharePoint Services\WssTasks.xml). The template should be global on the Sharepoint server.
   

<Portal>
        <site template="TFS-Project" language="1033" />
        ...
</Portal>



• Customize your TFS Project: Specify the target template for the TFS task to your new template. Since the Sharepoint project template (STP) already has the Project Server features activated, you don’t need to do that step. During project creation, TFS will add all of your default document libraries and documents. This includes whatever majik they do to make Excel and Project files in the template work with the proper Team Project.
• Publish the Project: but … don’t include the Project Workspace. See link referenced above. You will not get the option for "Do not create a workspace at this time” unless you do Setup step 1 (a step that Neil didn’t mention).
• Associate the Project Workspace with the TFS site: In the article referenced above, Neil details this quite well. I won’t repeat it here.
• Downside: Require manual association of the project workspace Sharepoint site with the TFS Sharepoint site.

Neither option uses undocumented extensibility points for TFS 2008. And yes, there are variations between the two that work – with unique sets of good/bad/ugly. This post is long enough without going into all of the possible permutations.

Very Important Disclaimer: While nothing here falls outside of the documented extensibility points for TFS 2008, I doubt that anything is in the product team’s upgrade test cases – especially since nothing here seems to be documented anywhere else.  I have not yet tested the upgrade path for either of these techniques to TFS 2010. Considering the (very frustrating) lack of guidance and documentation for these techniques, I cannot say anything one way or the other. I will, however, be testing and experimenting with the upgrade paths (Hyper-V shapshots ROCK!!!) and any goodness that I discover will be published here. Of course, I’m more than happy to work with the TFS product team on this …

There comes a time in every TFS implementation where you need to create your own set of default project work items in your template. You can do this straight in the XML by editing workitems.xml in your process template’s Work Item Tracking folder. I said you could, not that you’d actually want to! You can also use the Process Editor to add them one at a time like you do with the Team Foundation client. Uggh. Wouldn’t it be better just to use Excel to put them into a scratch, temporary TF Project? I certainly think so; I can enter work items in Excel a lot faster than I can in the TF Client (or in the Process Editor).

I did some looking about thinking that someone must have solved this already and posted some code, open source, whatever. There was one result that looked promising but … the URL didn’t work (so I’m not posting it here). <Heavy Sigh>

Oh well, time to pop open Visual Studio and start coding …

It turned out that it was pretty easy, in fact. In the sample below, you take a raw work item query (in WIQL) and get the list of work items that you want to export with the columns that you also want to export. The order doesn’t matter. From there, you execute the query against the Work Item Store to get the list of work items. Loop over them and write to XML in the format that the project template uses.

It is, for the most part, really that simple. There are only 2 things that you really have to watch out for. First, the project name should be replaced with a marker for the target project name  ($$PROJECTNAME$$). Second, you need to make sure that the field is applicable to the work item type. This is something that you need to be especially aware of if you have multiple work item types in your default work item set. With all of that … I am proud to present to you … the actual code (exception handling removed for clarity … put yours in!)

public void Export(
    string serverName, string projectName, string filePath, string WIQL)
{
    TeamFoundationServer tfs = TeamFoundationServerFactory.GetServer(
                            serverName, 
                            new UICredentialsProvider() );
    tfs.Authenticate();
    WorkItemStore store = 
        (WorkItemStore)tfs.GetService(typeof(WorkItemStore));
    WorkItemCollection workItems = store.Query(WIQL);
    using (System.IO.Stream output = 
        System.IO.File.Open(filePath, 
            System.IO.FileMode.Create, 
            System.IO.FileAccess.ReadWrite))
    {
        using (XmlWriter writer = XmlWriter.Create(output))
        {
            writer.WriteStartDocument(true);
            writer.WriteStartElement("WORKITEMS");
            WriteWorkItems(projectName, workItems, writer);
            writer.Flush();
            writer.Close();
        }
        
    }
}

private static void WriteWorkItems(string projectName, 
    WorkItemCollection workItems, XmlWriter writer)
{
    for (int i = 0; i < workItems.Count; i++)
    {
        var workItem = workItems[0];
        writer.WriteStartElement("WI");
        writer.WriteAttributeString("type", workItem.Type.Name);
        foreach (FieldDefinition field in workItems.DisplayFields)
        {
            if (workItem.Fields.Contains(field.Name))
            {
                writer.WriteStartElement("FIELD");
                writer.WriteAttributeString("refname", field.ReferenceName);
                writer.WriteAttributeString("value", 
                    workItem.Fields[field.Name].Value.ToString().Replace(
                            projectName, "$$PROJECTNAME$$"));
                writer.WriteEndElement();
            }
        }
        writer.WriteEndElement();
    }
}

Now, I don’t know about you, but when I see “God Mode”, I think of running around in some shooter killing all the bad guys/monsters/aliens/whatever without any worry of damage or harm coming to me. Those of you that cut your teeth on some of the old school DOS-based games of yester-century (a la Doom) will know exactly what I’m talking about.

I don’t think of “God Mode” when it comes to an operating system. Unless, perhaps, running everything as admin. Really, though, that’s probably more akin to anti-God Mode if you think about it; you need to be a little more worried about potential damage and hazards since you are running as admin.

But then there’s Windows 7 and we have things like User Account Control to keep you from doing too much harm. So is “God Mode” some super-anti-UAC thingie that makes it like it was in the old days when you could call deltree from the root of your C: drive and it would happily go about doing it? (Note: Yes, I know someone that did this. No, it wasn’t me. And no, I don’t recommend trying it.)

Nope, not at all like the God Mode that I knew and loved from the days of Doom.

So when I saw an article called “Understanding Windows 7's 'GodMode'”, I was intrigued. It’s not the “God Mode” of Doom-yore, but it is interesting nonetheless and I’m kinda liking it. You create a folder and name it “GodMode.{ED7BA470-8E54-465E-825C-99712043E01C}” (no quotes, of course). Then … behold! the icon changes and you have access to a ton of controls and settings for the operating system. It does work on Windows Server 2008 R2 x64, which is what I’m running right now.

It reminds me, in a way, of the Windows 95 Product Team Easter Egg. But simpler to create and a lot more useful.

One of the things that I’m working on as a part of the Team Foundation implementation that I’m working on is to create a series of short (10-15 minute … for me, that’s short) videos on different topics related to working with TFS within the environment. If these were generalized TFS videos, it wouldn’t be an issue but they aren’t – they are highly customized to the specific implementation and the custom template that we’re using. As a part of it, I needed to create a virtualized environment that reproduces the production environment that I could use for the videos (as well as testing). Since I want this to be as close as possible to the actual, working, production environment, I wanted to use the same URLs as we use in production. Setting up and resolving the URLs is easy enough … it’s all a private Hyper-V environment that has its own domain controller, DNS and DHCP. Adding credentials for invisible, integrated authentication was easy as well. Or so I thought … it turned out that it wasn’t as simple as I would have thought.

The URLs were all set up. I used tfsadminutil configureconnections to set the URLs in TFS so that the registration worked correctly. I cleared the TF client cache. Yet … it still didn’t work. I got errors from the client (running on the Hyper-V host system) that basically said “ummm … so sorry, it’s not working”. Looking at the app tier’s event log, I saw a series of errors including the following (this proved to be the important one):

Detailed Message: TF53002: Unable to obtain registration data for application VersionControl.
Web Request Details
    Url: http://apptier.my.testurl.com/VersionControl/v1.0/repository.asmx [method: POST]
    User Agent: Team Foundation (devenv.exe, 9.0.30729.1)
    Headers: Content-Length=319&Content-Type=application%2fsoap%2bxml%3b+charset%3dutf-8&Accept-Encoding=gzip&Accept-Language=en-US&Expect=100-continue&Host=apptier.my.testurl.com&User-Agent=Team+Foundation+(devenv.exe%2c+9.0.30729.1)&X-TFS-Version=1.0.0.0&X-TFS-Session=09b73cf7-6f05-4528-b8d7-2b531d43a409
    Path: /VersionControl/v1.0/repository.asmx
    Local Request: False
    Host Address: 10.5.5.1
    User: TESTDOMAIN\JSawyer [authentication type: NTLM]

Exception Message: TF30063: You are not authorized to access apptier.my.testurl.com.80. (type TeamFoundationServerUnauthorizedException)

I would get this same error regardless of client (VS or TFS Web). The account being used was a domain administrator … so that ruled out that it was really an authorization issue. Besides that, it all worked find until I did all the mucking about with the URLs.

Looking about the Internets, I found a blog entry from Nick Berardi gave the solution (and yes, our current production environment in Win2K3)… I set the TFS URLs in the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\DisableLoopbackCheck registry key, rebooted and voila!! It all worked!