MSBuild and Multiple Environments
Even if you are the kind of developer who rides into work on the back of a raging bull wearing one of your many cowboy hats whistling your favourite Willie Nelson tune, it is very likely you have multiple environments for your software. At the very least you have your Developer and Production environments, hopefully you also have QA and perhaps BUILD and UAT.
Each of these environments requires subtle changes (usually to a config file) that must be made when you create a release/build, e.g. database connection string.
So this post is about one technique you can use with MSBuild to simplify the process of creating a release for these different environments. Using this technique also makes creating releases much faster and less error prone. It has a ClickOnce flavour but the core concepts are relevant for any method you use to promote new versions of your software.
Once you’ve read this posting, checkout this one for an extension of the ideas presented here and a subtly different way of creating your releases using a ‘Master Build’ approach.
Creating a release is just one phase of the development lifecycle. We need to ensure that any approach we take to simplify the release process does not interfere with other phases. These are the scenario’s that our approach must work seamlessly and autonomously for:-
- Developer, get the latest version, build and debug the app.
- Build machine detects a change, get latest version, build, run tests, create successful build.
- Publish (via ClickOnce) a successful build to each environment using a batch file (or other similar, not interactive process).
- Developer needs to debug a non DEV environment.
- Run tests on Developer machine and Build box.
Let’s assume for this posting that we have these environments: DEV, BUILD, QA, PROD
Setting Up Your Customizations
Now in your new Customized.targets file add this to the top (if you haven’t already)
<PropertyGroup> <BuildEnvironment>DEV</BuildEnvironment> </PropertyGroup>
Then directly underneath add:
<Choose> <When Condition=" '$(BuildEnvironment)' == 'DEV' "> <PropertyGroup> <BaseUrlWebServices>https://mywebserviceaddress</BaseUrlWebServices> <PublishDir>\\MyMachine\ReleaseFolder\$(BuildEnvironment)\</PublishDir> </PropertyGroup> </When> <When Condition=" '$(BuildEnvironment)' == 'QA' "> <PropertyGroup> <BaseUrlWebServices>https://mywebserviceaddress</BaseUrlWebServices> <PublishDir>\\MyMachine\ReleaseFolder\$(BuildEnvironment)\</PublishDir> </PropertyGroup> </When> </Choose>
(for brevity I have only put DEV and QA. BUILD and PROD should be there to).
You are now set up and ready to go with customizations for different environments ‘within’ MSBuild. So lets discuss what we have done here and how we’re going to use it.
New Property – BuildEnvironment
This is a brand new property that MS Build knows nothing about. It is important to set the default here to DEV to deal with scenario 1 mentioned above…developer getting the latest version and debugging the app. For each of the other scenarios we can pass in the environment as a parameter to MSBuild, but more on that later.
Very simply we use the value of this parameter to set the value of a number of other variables.
In this section (from the xml above) we list all possible values for BuildEnvironment and set other variables that are environment specific. Easy :-)
As an example I’ve included one other new property, BaseUrlWebServices. I’m going to use this property to update my app.config later. Note that these properties can be brand new ones or can be ones that MSBuild knows about and uses, just double check any values that MSBuild knows about because I have noticed that some are successfully set using this approach but some failed to be set. E.g the PublishUrl property works but InstallUrl does not (didn’t investigate why). (side note -The value InstallUrl may be changed by passing it in as a paramater to MSBuild).
The other example property is a built in MSBuild property, the PublishDir used by ClickOnce to determine where to publish a release. In the example above it has the same value for both environments, but uses the BuildEnvironment variable as the destination sub directory.
Using The Values
Ok so now we have these different values set up for all of our environments, how do we use them?
PublishDir is sorted already, MSBuild will automatically pick it up when publishing.
BaseUrlWebServices we want to use to update values in our app.config file.
Update App.Config with Environment Specific Data
First we need to change our app.config file so that is has a value we can consistently update.
So where you have something like this:-
Replace it with this:-
Now the tricky bit. At what ‘stage’ in the build/publish process do we update the app.config file? Ideally we don’t want to update the app.config file at all, what we want to update is the config file that gets created in the bin directory, Myapp.exe.config. Why? Well if you update the app.config file then the update will only succeed once…the keys will all be updated with their respective replacement values. While this is kind of OK the main problem is this scenario; developer gets latest version, runs the app (which updates the config file) then modifies the config file (adds a new key value pair) then checks in the change which now not only includes their new key value pair but also the environment keys are now values.
For ClickOnce, we also want to ensure that the changes are applied BEFORE the manifests are created, otherwise the application will not install (because a file has been modified).
So given this list of possible targets to override in MSBuild, which should we use? I use two, this is my approach and why….
1. Override BeforeBuild and AfterBuild
<Target Name="BeforeBuild" DependsOnTargets="ReleaseModeUpdateConfig" /> <Target Name="AfterBuild" Condition=" '$(Configuration)' == 'Debug' " DependsOnTargets="UpdateConfigFileForEnvironment" />
I have removed a couple of extra ‘DependsOnTargets’ items which are specific to my project. It is important to note that everything I do in AfterBuild I only want to happen when in Debug mode but for BeforeBuild I have some targets I want to run only in ReleaseMode and some also in Debug mode.
2. Create ReleaseModeUpdateConfig
<Target Name="ReleaseModeUpdateConfig" Condition=" '$(Configuration)' == 'Release' " DependsOnTargets="UpdateConfigFileForEnvironment" />
The only reason this has its own target is so that I can apply the Condition to ensure it only gets run in Release mode. I only want it to run in Release mode because when I Publish it is always in Release mode and in day to day development most developers will work in Debug mode, so the update of the config file won’t interfere with their work.
I would prefer to use this property GenerateClickOnceManifests, it gets set to true when you call the Publish target…but for some reason it is always true, even when only running the Build target.
3. Create the UpdateConfigFileForEnvironment target
<Target Name="UpdateConfigFileForEnvironment"> <PropertyGroup> <AppConfigFileName>$(TargetPath).config</AppConfigFileName> <AppConfigFileName Condition=" '$(Configuration)' == 'Release' ">app.config</AppConfigFileName> </PropertyGroup> <!-- Update Service address --> <FileUpdate Files="$(AppConfigFileName)" Encoding="ASCII" Regex="SomeUniqueKeyWithAGoodName" ReplacementText="$(BaseUrlWebServices)" /> </Target>
I create a new property here, AppConfigFileName which controls which file actually gets updated. The build mode drives this so if we are in Release mode then we update the app.config directly but if we are in any other mode then we update the config file that gets created in the bin directory….myApp.exe.config.
The FileUpdate section requires msbuild community tasks and simply replaces the keys (in this case, SomeUniqueKeyWithAGoodName), with the environment specific value.
Before and After
When publishing it is very important that the app.config update occurs in the BeforeBuild override because the creation of the manifests occurs during CoreBuild, so AfterBuild is too late. Note also that MSBuild will always use the app.config file that is in your .csproj directory when creating the manifests.
AfterBuild is used when we’re updating myApp.exe.config because if you start with a clean bin directory the file won’t exist in BeforeBuild.
ClickOnce Deployment With Our Changes
Ok so we have these environment specific information setup, now what do we do, how do we use it to create a new release?
Create a batch file (in your csproj directory), called publishQA.bat, in it put this:-
msbuild /t:Publish /p:Configuration=Release /p:BuildEnvironment=QA /p:ApplicationVersion=18.104.22.168
(of course you need to change your version number). What we’re doing here is simply telling msbuild to run the ‘Publish’ target, ensuring that it builds in Release mode and we pass in our verion number. Importantly we also tell it which environment we are publishing for so MSBuild will pick up our environment specific information.
Create a similar file for each environment then when you want to publish to QA all you need to do is open up a Visual Studio command prompt (in the directory with the batch files), then type publishQA -> Enter….a new release will be created for the QA environment…with all of the correct environment specific information!
Build Box Scenario
Making this work with the Build box scenario should be relatively simple. All we need to do is ensure the BuildEnvironment property gets set. A couple of possible solutions are:-
- Make the BuildEnvironment value come from an Environment Variable on the machine
- Or set the BuildEnvironment after the build process ‘gets’ the latest version when a change is detected
Run Tests Scenario
The problems you will face in this scenario will vary greatly depending on how you have things set up. I can only describe what I do, the challenges faced with my setup and how to make it work.
- A separate test project for each assembly
- Each project which requires an app.config has one but I never modify these files directly
- I only have one ‘master’ config file (which lives in the main projects directory, i.e. the one that is usually set as the startup project)
- All projects that need a config file, e.g. Test projects, have a pre-build event which copies (XCOPY) the config file from the ‘master’ config file location
For me the key advantages of this setup is simplicity and maintainability. I know I only have one config file that I need to modify for everything.
Ok so before going on, let me clarify something. Most Test projects won’t care about the config file because they should be testing the logic and using mock objects etc. However I like to have some ‘Integration‘ unit test projects that do care about the contents of the config file and which server they are talking to etc.
The PROBLEM with this setup is with the environmental specific info. With the current setup the raw config file isn’t valid for use until my ‘main’ project is built so my test projects have a problem. Or do they? We definately do not want to duplicate the data and logic that is in the Customized.targets file so how do we ensure our config file is getting the correct values for these integration test projects?
<Import Project="..\..\MyApp\Customized.targets" /> <Import Project="$(MSBuildExtensionsPath)\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets" />
Just import Customized.targets into the integration test projects as well! (Note that you must also import the MSBuild Community tasks targets). This means that when a developer runs the tests on his or her machine in Debug mode they will run successfully and also when the Build Box builds and runs the tests in Release mode it will also work…and be run against the correct environment.
This is the pre-build event that pulls the config file from the ‘main’ project to the test project…
XCOPY ..\..\..\..\MyApp\app.config $(ProjectDir) /R /Y
There are many ways you could potentially solve the problem of environment specific information. I like this approach for these reasons:-
- All the environment specific information is in one file
- Unless they work in Release mode, the approach does not interfere with Developers getting on with their work
- Creating new releases to any environment is quick, easy and reliable.
I’m always looking for ways of improving processes so please do point out any flaws you find or means of improving this approach. If you are using a completely different approach I’d be very interested in hearing about it and what you see as the pro’s and con’s of that approach vs this one.
Happy building! :-)