Archive for the ‘ Uncategorized ’ Category

2010 in review

The stats helper monkeys at mulled over how this blog did in 2010, and here’s a high level summary of its overall blog health:

Healthy blog!

The Blog-Health-o-Meter™ reads This blog is on fire!.

Crunchy numbers

Featured image

The average container ship can carry about 4,500 containers. This blog was viewed about 20,000 times in 2010. If each view were a shipping container, your blog would have filled about 4 fully loaded ships.


In 2010, there were 8 new posts, growing the total archive of this blog to 32 posts. There were 19 pictures uploaded, taking up a total of 483kb. That’s about 2 pictures per month.

The busiest day of the year was February 5th with 153 views. The most popular post that day was Software Project Failure.

Where did they come from?

The top referring sites in 2010 were,,,, and

Some visitors came searching, mostly for wcf concurrencymode.

Attractions in 2010

These are the posts and pages that got the most views in 2010.


Software Project Failure February 2010


ClickOnce – Creating Publish Page from msbuild December 2009


MSBuild and Multiple Environments December 2009


Silverlight Communication Exception March 2009


SVN Merge Without Conflicts March 2009


ClickOnce Master Build

Previously I have posted about how to publish a ClickOnce release for multiple environments. Whilst this works well there are two reasons (that I can think of) why it is not appropriate or good enough.

First, you or your manager may be a bit of a purist in terms of releasing to Production ‘exactly’ what has been tested, specifically the exact same compiled assemblies.

Second, you may need to create a release for several different customers who each have several of their own ‘environments’ but you do not want to provide them with your source code.

Third…both of the above. 🙂

When you ‘Publish’ with MsBuild it requires the source code and it recompiles your assemblies. So how can we easily create releases for multiple environments but only Publish once? This posting offers one solution which has worked for me in the past, it’s not a complete ‘how to’ guide I just wanted to cover the idea and problems I ran into along the way.

Master Build Environment

Make sure you take a look at my previous posting as this process builds on those ideas.

One of the final steps of this process is to re-sign the setup.exe, however in order for that final step to work you have to change one of your ClickOnce deployment settings in your .csproj file.


SignManifests must be false! Otherwise the signtool gets confused when re-signing the setup.exe. Check this out for a bit more info on this known issue.

So the first thing to do is create a BuildEnvironment called Master. The configuration details of this (web service addresses or database connnection strings etc) should point to nowhere, i.e. if someone deployed this build then it should not work. Publishing this is easy just follow the steps defined in the multiple environment posting, the process of creating a release with environment specific information that is tricky.

MsBuild Changes for Master Build Publish

When you publish with ClickOnce it creates a folder ‘Application Files’ where it puts the contents of the release. The space in that file name causes a problem with MAGE when re-signing the app. Now the only way I could see to change this was to cut and paste the _CopyFilesToPublishFolder target from Microsoft.Common.Targets and modifying the _DeploymentApplicationFolderName property in my version to remove the space. I think it is a really BAD idea to change the Microsoft.Common.Targets directly so that option was ruled out immediately and as you can see in the code below, Application Files (version below already has the space removed) is a hardcoded value, not a settable property…therefore even though it is ugly, cut and paste seems the only viable solution.

	  This Target code has been cut and paste from Microsoft.Common.Targets.
	  Only ONE thing has been changed.
	  Application Files changed to ApplicationFiles (i.e. the space removed)
	  The only other way of changing that value is to modify Microsoft.Common.Targets
	  which is a really bad idea (would have to ensure this was changed on ALL machines.)

	  The space causes a problem when re-signing the app with MAGE

        <!-- Compute name of application folder, which includes the assembly name plus formatted application version.
             The application version is formatted to use "_" in place of "." chars (i.e. "1_0_0_0" instead of "").
             This is done because some servers misinterpret "." as a file extension. -->
        <FormatVersion Version="$(ApplicationVersion)" Revision="$(ApplicationRevision)" FormatType="Path">
            <Output TaskParameter="OutputVersion" PropertyName="_DeploymentApplicationVersionFragment"/>


        <!-- Copy files to publish folder -->
        <FormatUrl InputUrl="$(_DeploymentApplicationUrl)">
            <Output TaskParameter="OutputUrl" PropertyName="_DeploymentFormattedApplicationUrl"/>
        <FormatUrl InputUrl="$(_DeploymentComponentsUrl)">
            <Output TaskParameter="OutputUrl" PropertyName="_DeploymentFormattedComponentsUrl"/>

Note that the space only causes a problem with MAGE.exe, it does not cause a problem with MAGEUI.exe! MAGEUI.exe happily re-signs the release even with the space in the directory name. We cannot use MAGEUI.exe however because this process needs to be automated, if you use MAGEUI.exe someone has to run it and manually set the values…not good enough.

It took a while to figure out that the space in the dir name was causing a problem so I hope this saves someone else some time…

Creating a Release

Ok so now we have a published build that no one can use, the first place we want to release to will be QA, then from there UAT so what we need now is an automated way of creating a release configured for these environments.

Creating a release should be as easy as publishing a build, it should be a one step process.

Create a batch file ‘createRelease.bat’ which performs all of the necessary steps. Those steps are:

  1. Create or clean the ‘working’ directory.
  2. Copy the master build into the working directory.
  3. Run the MsBuild steps.
  4. Create a zip file of the new release. (not necessary just makes it a little easier to move the releases around)

Run the MsBuild Steps

Add a new target in your customized.targets file, ModifyMasterBuildForEnvironment.

<Target Name="ModifyMasterBuildForEnvironment" DependsOnTargets="SetPropertyValues; ConfigureForEnvironment; RecreateManifests" />


There are a few properties which need to be set for the resigning process.

	<Target Name="SetPropertyValues" >
		<Message Text="FullReleasePath $(FullReleasePath)" />

		<MSBuild.ExtensionPack.Framework.TextString TaskAction="Replace" OldString="$(ApplicationVersion)" OldValue="." NewValue="_">
			<Output PropertyName="buildX" TaskParameter="NewString"/>



Add your own target(s) here that does the work to configure the release for a given environment. e.g. update the config file setting database connection strings or web service urls.


Now that you have modified the published release it is no longer a valid ClickOnce release. Try to install it and it will fail. What you need to do is update and re-sign the manifests (the Application Manifest and the Deployment Manifest). You also need to update and resign the Setup.exe!

	<Target Name="RecreateManifests">

		<!-- Application manifest update and resign -->
		<Message Text="Removing .deploy suffix..." />
		<RenameFiles DirectoryPath="$(MSBuildProjectDirectory)\$(FullReleasePath)" RemoveSuffix=".deploy" IncludeSubDirectories="True" />
		<Message Text="Updating application manifest" />
		<Exec Command="mage -Update $(ApplicationManifestPath) -FromDirectory $(FullReleasePath) -ToFile $(ApplicationManifestPath)" />
		<Message Text="Adding .deploy suffix..." />
		<RenameFiles DirectoryPath="$(MSBuildProjectDirectory)\$(FullReleasePath)" AppendSuffix=".deploy" IncludeSubDirectories="True" ExcludedFiles="$(ApplicationManifestName)" />
		<Message Text="ApplicationManifestPath $(ApplicationManifestPath)" />
		<Exec Command="mage -Sign $(ApplicationManifestPath) -CertFile $(CertFileName)" />

		<!-- Deployment manifest update and resign -->
		<Message Text="Updating Deployment manifest (ProviderFullUrl=$(ProviderFullUrl))" />
		<Exec Command="mage -Update $(DeploymentManifestPath) -AppManifest $(ApplicationManifestPath) -ProviderUrl $(ProviderFullUrl)" />
		<Message Text="DeploymentManifestPath $(DeploymentManifestPath)" />
		<Exec Command="mage -Sign $(DeploymentManifestPath) -CertFile $(CertFileName) " />

		<!-- Setup.exe update and resign -->
		<Message Text="Updating $(WorkingDirectory)\Setup.exe..." />
		<Exec Command="$(WorkingDirectory)\Setup.exe /url=$(ProviderBaseUrl)" />
		<Message Text="Signing $(WorkingDirectory)\Setup.exe..." />
		<Exec Command="signtool sign /v /f $(CertFileName) $(WorkingDirectory)\Setup.exe" />


Note that before Updating the Application Manifest we have to remove the .deploy extension from the file names, then add the .deploy extension back onto the file names before Signing. (if you have configured ClickOnce to NOT append .deploy then obviously just remove these steps).

RenameFiles Task

To get the renaming behaviour I wanted I had to create my own MsBuild task. You will need to do the same, it’s a little crude as I had limited time but this is the simple little task I created….

    public class RenameFiles : Task
        public override bool Execute()
            var directory = new DirectoryInfo(DirectoryPath);
            if(directory == null)
                throw new ApplicationException(string.Format("Directory path is invalid: {0}", DirectoryPath));
            Console.WriteLine(string.Format("DirectoryPath:{0}", DirectoryPath));


            return true;

        private void ProcessDirectory(DirectoryInfo directory)
            if(! IncludeSubDirectories)

            var subs = directory.GetDirectories();
            foreach (DirectoryInfo sub in subs)

        private void RenameAllFiles(DirectoryInfo directory)
            var files = directory.GetFiles();

            if(files.Length == 0)

            foreach (FileInfo file in files)
                string fileName = file.Name;
                if (ExcludedFileList.Exists(name => name.Equals(fileName, StringComparison.InvariantCultureIgnoreCase)))
                    Console.WriteLine(string.Format("{0} is excluded from renaming.", file.Name));
                Console.WriteLine(string.Format("Renaming {0}", file.Name));

        private void RemoveExtensions(FileInfo file)
            if (string.IsNullOrEmpty(RemoveSuffix))
            file.MoveTo(file.FullName.Replace(RemoveSuffix, string.Empty));

        private void AppendExtensions(FileInfo file)
            if (string.IsNullOrEmpty(AppendSuffix))
            file.MoveTo(string.Format("{0}{1}", file.FullName, AppendSuffix));

        public string DirectoryPath { get; set; }
        public string RemoveSuffix { get; set; }
        public string AppendSuffix { get; set; }
        public string ExcludedFiles { get; set; }
        public bool IncludeSubDirectories { get; set; }

        private List<string> ExcludedFileList
                if (excludedFileList == null)
                    excludedFileList = CommonFunctions.CreateListFromCommaSeparatedString(ExcludedFiles);
                return excludedFileList;
        private List<string> excludedFileList;


There are a number of other little steps we do to make this release process work here but these are really specific to how we want it to work. What I have outlined above is the core steps involved including some problems you will face and their solutions. I hope this posting helps you get over the most challenging of the hurdles you will face in setting a smooth ‘Master Build’ release process but there is still alot of extra steps you will need to create yourself to get it working end to end.

One thing I would recommend is, once you have a working createRelease.bat (note it doesn’t have to be a batch file, could be an exe or powershell or whatever works best in your situation) then create a createReleaseALL.bat file. In there make n calls to createRelease.bat…one for each environment you need to create releases for. That way if you have a dozen environments you can still create a release for all of them with a one step process 🙂

I have had to do this fairly quickly so if something is not clear please let me know and I will see if I can improve the post over time.

Final note – It does take time to set this up and get it working correctly, but it is so worth it! Over time you will save much more time than you spent setting it up, guaranteed!

Additional Resources

A few links which helped me to piece this puzzle together…

ClickOnce with bootstraper setup.exe need to change URL without any build


Batch File Error

Weird Characters Causing Batch File Error

is not recognized as an internal or external command, operable program or batch file

That’s the error I was getting, weird considering the only thing (after removing everything else) in the file was @ECHO OFF. The ‘weird’ characters are shown in the little snippet below

After a bit of head scratching I opened notepad++, switched on ‘Show all characters’…nothing strange there. I had seen this before but I can never remember….then I did! Remember that is, the problem was the file, when it was created (by visual studio…as a txt file), was created with UTF-8 encoding, needs to be ANSI for batch files (or at least it can’t be UTF-8).

Green indicates ‘good’ encoding for batch files, red bad.

With notepad++ you can see what the encoding is (under Format) and change it. I changed my batch file to ANSI and voila it works!

I’ve blogged it so I can remember next time 🙂

MSBuild and Multiple Environments

Even if you are the kind of developer who rides into work on the back of a raging bull wearing one of your many cowboy hats whistling your favourite Willie Nelson tune, it is very likely you have multiple environments for your software. At the very least you have your Developer and Production environments, hopefully you also have QA and perhaps BUILD and UAT.

Each of these environments requires subtle changes (usually to a config file) that must be made when you create a release/build,  e.g.  database connection string.

So this post is about one technique you can use with MSBuild to simplify the process of creating a release for these different environments. Using this technique also makes creating releases much faster and less error prone. It has a ClickOnce flavour but the core concepts are relevant for any method you use to promote new versions of your software.

Once you’ve read this posting, checkout this one for an extension of the ideas presented here and a subtly different way of creating your releases using a ‘Master Build’ approach.


Creating a release is just one phase of the development lifecycle. We need to ensure that any approach we take to simplify the release process does not interfere with other phases. These are the scenario’s that our approach must work seamlessly and autonomously for:-

  1. Developer, get the latest version, build and debug the app.
  2. Build machine detects a change, get latest version, build, run tests, create successful build.
  3. Publish (via ClickOnce) a successful build to each environment using a batch file (or other similar, not interactive process).
  4. Developer needs to debug a non DEV environment.
  5. Run tests on Developer machine and Build box.

Let’s assume for this posting that we have these environments: DEV, BUILD, QA, PROD

Setting Up Your Customizations

Look at my other posting and do the pre-requisite (install MSBuild Community Tasks) and Step One.

Now in your new Customized.targets file add this to the top (if you haven’t already)


Then directly underneath add:

 <When Condition=" '$(BuildEnvironment)' == 'DEV' ">
 <When Condition=" '$(BuildEnvironment)' == 'QA' ">

(for brevity I have only put DEV and QA. BUILD and PROD should be there to).

You are now set up and ready to go with customizations for different environments ‘within’ MSBuild. So lets discuss what we have done here and how we’re going to use it.

New Property – BuildEnvironment

This is a brand new property that MS Build knows nothing about. It is important to set the default here to DEV to deal with scenario 1 mentioned above…developer getting the latest version and debugging the app. For each of the other scenarios we can pass in the environment as a parameter to MSBuild, but more on that later.

Very simply we use the value of this parameter to set the value of a number of other variables.


In this section (from the xml above) we list all possible values for BuildEnvironment and set other variables that are environment specific. Easy 🙂

As an example I’ve included one other new property, BaseUrlWebServices. I’m going to use this property to update my app.config later. Note that these properties can be brand new ones or can be ones that MSBuild knows about and uses, just double check any values that MSBuild knows about because I have noticed that some are successfully set using this approach but some failed to be set. E.g the PublishUrl property works but InstallUrl does not (didn’t investigate why). (side note -The value InstallUrl may be changed by passing it in as a paramater to MSBuild).

The other example property is a built in MSBuild property, the PublishDir used by ClickOnce to determine where to publish a release. In the example above it has the same value for both environments, but uses the BuildEnvironment variable as the destination sub directory.

Using The Values

Ok so now we have these different values set up for all of our environments, how do we use them?

PublishDir is sorted already,  MSBuild will automatically pick it up when publishing.

BaseUrlWebServices we want to use to update values in our app.config file.

Update App.Config with Environment Specific Data

First we need to change our app.config file so that is has a value we can consistently update.

So where you have something like this:-

<endpoint address=""

Replace it with this:-

<endpoint address="SomeUniqueKeyWithAGoodName/UserService.svc"

Now the tricky bit. At what ‘stage’ in the build/publish process do we update the app.config file? Ideally we don’t want to update the app.config file at all, what we want to update is the config file that gets created in the bin directory, Myapp.exe.config. Why? Well if you update the app.config file then the update will only succeed once…the keys will all be updated with their respective replacement values. While this is kind of OK the main problem is this scenario; developer gets latest version, runs the app (which updates the config file) then modifies the config file (adds a new key value pair) then checks in the change which now not only includes their new key value pair but also the environment keys are now values.

For ClickOnce, we also want to ensure that the changes are applied BEFORE the manifests are created, otherwise the application will not install (because a file has been modified).

So given this list of possible targets to override in MSBuild, which should we use? I use two, this is my approach and why….

1. Override BeforeBuild and AfterBuild

<Target Name="BeforeBuild"
    DependsOnTargets="ReleaseModeUpdateConfig" />
<Target Name="AfterBuild"
    Condition=" '$(Configuration)' == 'Debug' "
    DependsOnTargets="UpdateConfigFileForEnvironment" />

I have removed a couple of extra ‘DependsOnTargets’ items which are specific to my project. It is important to note that everything I do in AfterBuild I only want to happen when in Debug mode but for BeforeBuild I have some targets I want to run only in ReleaseMode and some also in Debug mode.

2. Create ReleaseModeUpdateConfig

<Target Name="ReleaseModeUpdateConfig"
    Condition=" '$(Configuration)' == 'Release' "  
    DependsOnTargets="UpdateConfigFileForEnvironment" />

The only reason this has its own target is so that I can apply the Condition to ensure it only gets run in Release mode.  I only want it to run in Release mode because when I Publish it is always in Release mode and in day to day development most developers will work in Debug mode, so the update of the config file won’t interfere with their work.

I would prefer to use this property GenerateClickOnceManifests, it gets set to true when you call the Publish target…but for some reason it is always true, even when only running the Build target.

3. Create the UpdateConfigFileForEnvironment target

<Target Name="UpdateConfigFileForEnvironment">

   <AppConfigFileName Condition=" '$(Configuration)' == 'Release' ">app.config</AppConfigFileName>

 <!-- Update Service address  -->
   ReplacementText="$(BaseUrlWebServices)" />

I create a new property here, AppConfigFileName which controls which file actually gets updated. The build mode drives this so if we are in Release mode then we update the app.config directly but if we are in any other mode then we update the config file that gets created in the bin directory….myApp.exe.config.

The FileUpdate section requires msbuild community tasks and simply replaces the keys (in this case, SomeUniqueKeyWithAGoodName), with the environment specific value.

Before and After

When publishing it is very important that the app.config update occurs in the BeforeBuild override because the creation of the manifests occurs during CoreBuild, so AfterBuild is too late. Note also that MSBuild will always use the app.config file that is in your .csproj directory when creating the manifests.

AfterBuild is used when we’re updating myApp.exe.config because if you start with a clean bin directory the file won’t exist in BeforeBuild.

ClickOnce Deployment With Our Changes

Ok so we have these environment specific information setup, now what do we do, how do we use it to create a new release?


Create a batch file (in your csproj directory), called publishQA.bat, in it put this:-


msbuild /t:Publish /p:Configuration=Release /p:BuildEnvironment=QA /p:ApplicationVersion=

(of course you need to change your version number).  What we’re doing here is simply telling msbuild to run the ‘Publish’ target, ensuring that it builds in Release mode and we pass in our verion number. Importantly we also tell it which environment we are publishing for so MSBuild will pick up our environment specific information.

Create a similar file for each environment then when you want to publish to QA all you need to do is open up a Visual Studio command prompt (in the directory with the batch files), then type publishQA -> Enter….a new release will be created for the QA environment…with all of the correct environment specific information!

Build Box Scenario

Making this work with the Build box scenario should be relatively simple. All we need to do is ensure the BuildEnvironment property gets set. A couple of possible solutions are:-

  • Make the BuildEnvironment value come from an Environment Variable on the machine
  • Or set the BuildEnvironment after the build process ‘gets’ the latest version when a change is detected

Run Tests Scenario

The problems you will face in this scenario will vary greatly depending on how you have things set up. I can only describe what I do, the challenges faced with my setup and how to make it work.

My setup:-

  1. A separate test project for each assembly
  2. Each project which requires an app.config has one but I never modify these files directly
  3. I only have one ‘master’ config file (which lives in the main projects directory, i.e. the one that is usually set as the startup project)
  4. All projects that need a config file, e.g. Test projects, have a pre-build event which copies (XCOPY) the config file from the ‘master’ config file location

For me the key advantages of this setup is simplicity and maintainability. I know I only have one config file that I need to modify for everything.

Ok so before going on, let me clarify something. Most Test projects won’t care about the config file because they should be testing the logic and using mock objects etc. However I like to have some ‘Integration‘ unit test projects that do care about the contents of the config file and which server they are talking to etc.

The PROBLEM with this setup is with the environmental specific info. With the current setup the raw config file isn’t valid for use until my ‘main’ project is built so my test projects have a problem. Or do they? We definately do not want to duplicate the data and logic that is in the Customized.targets file so how do we ensure our config file is getting the correct values for these integration test projects?


<Import Project="..\..\MyApp\Customized.targets" />
<Import Project="$(MSBuildExtensionsPath)\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets" />

Just import Customized.targets into the integration test projects as well! (Note that you must also import the MSBuild Community tasks targets). This means that when a developer runs the tests on his or her machine in Debug mode they will run successfully and also when the Build Box builds and runs the tests in Release mode it will also work…and be run against the correct environment.

This is the pre-build event that pulls the config file from the ‘main’ project to the test project…

XCOPY ..\..\..\..\MyApp\app.config $(ProjectDir) /R /Y


There are many ways you could potentially solve the problem of environment specific information. I like this approach for these reasons:-

  • All the environment specific information is in one file
  • Unless they work in Release mode, the approach does not interfere with Developers getting on with their work
  • Creating new releases to any environment is quick, easy and reliable.

I’m always looking for ways of improving processes so please do point out any flaws you find or means of improving this approach. If you are using a completely different approach I’d be very interested in hearing about it and what you see as the pro’s and con’s of that approach vs this one.

Happy building! 🙂

TFS Unlock Exclusive Checkout

I’ve been using TFS for the project I have been working on these past few months. I have to say that after using SVN that I’m not loving TFS, although to be fair we’re mainly using TFS much like source safe, i.e. as a repository without making much use of the work items etc.

The main thing I don’t like about it is it gets in your way. When working with SVN as the repo you work completely disconnected, make your changes then submit them when you are ready…it’s only then that you have to interact with SVN. TFS on the other hand still has this old school idea of checking items out which means every time you change a file you have to interact with TFS, usually not a problem but gets annoying when the repo regularly takes 5-10 seconds to peform the checkout.

The Problem

So today I discovered a feature in TFS that I like – removing a lock on an exclusively checked out file.

You know the issue, you want to work on a file but your colleague has the file checked out exclusively and you get the error, blah.cs – File is exclusively checked out by another user.

Then he can’t check it in because then he’d have to check in everything else and it’s not working right now…and he can’t undo the checkout because there are many changes…blah blah blah.

The Solution

Well I assumed that TFS would be like SourceSafe and these were really the only options but NO! TFS provides a neat little third option…unlock!


The guy who has the file exclusively checked out has to do the unlocking of course but all he has to do is right click on the file in solution explorer, click unlock and voila….you can now also check out the file. Too easy!

Of course you have to have multiple checkouts turned on on the TFS server, seriously though, if you do not have multiple checkouts turned on…..WHY???

Ode to Michael Jackson

Composed almost entirely of song titles….can you count how many?

2 Bad Michael you are Gone Too Soon…seems Heaven Can(t) Wait, Whatever Happens despite your Bad Monkey Business you were always Invincible.

I’ll always Remember The Time I saw the Man In The Mirror, the Smooth Criminal Burn This Disco Out on the HIStory tour. Speechless I had a huge Smile and Butterflies in my stomach…You Rock My World.

Tell me MJ, Is It Scary, is it Dangerous, are there Ghosts…are there Just Good Friends or is there Someone In the Dark? I’m sure You Are Not Alone.

Whatever Happens don’t Cry…Jam at 2000 Watts until the Break Of Dawn and there’s Blood On The Dance Floor…you were a Thriller!