Tuesday, May 31, 2011

TFS Build Workflow

This will be a rather lengthy post. I've received several emails asking how to implement the TFS build scripts. Well, it's pretty easy actually.

First things first. The build scripts need to be available to your build agent, of course. To that end, you can either put them in a fixed location and call them from that fixed path. However, at Streamline Systems we actually sometimes switch between building on a dedicated build VM (per client - to support different versions of AX) to building on an AOS server where consultants can remote in to test.
To support our multiple scenarios, we actually put the build scripts IN the source tree, so that the build script workflow can run it, regardless of what server or where the scripts are located. They're always in the source's root. The added benefit is you're versioning your build scripts, in case you need something crazy for a particular project/branch.

Anyway, the following is a TFS project based on the CMMI template. I'm unsure if the Agile template has a different build template. I'm assuming it's slightly different, except for these essentials steps, regardless of your project management style.

First, locate the spot in the XAML (have fun scrolling) where it actually does the compiling. I've noticed there's more than one MSBuild statement in there, so locate this exact spot.



Now, we want to get rid of the MSBuild task (delete!) and add the calls to our build scripts instead. To get the console output, errors and warnings to show up in the logs, we need to capture the output. So, to that end, add a step (where the MSBuild used to be) and add the "InvokeProcess" workflow step. In fact, add two while you're at it.
Now, as you may have noticed, the build scripts have some optional parameters. The cool thing with TFS is, you can create "arguments". That way, even if you don't use the optional parameters NOW, you can still use them later. Also, we'll use arguments to make the AX configuration file a parameter to the build script.
As for the standard and error output, you can have it write to the actual output, and then use them to write as build output / build errors. Completed, it should look something like this.



As you can tell, the process you invoke, should be "powershell.exe". We use the SourcesDirectory as the working directory, since that is where our build scripts will be located (TFS will pull them from the source tree, along with the XPOs). As for the arguments, we pass in the main build script name along with the properties.



You can see we added an "argument" for the config file which you can then use in the parameters to the build script. As for the second InvokeProcess step you've added, that is used to get the compiler errors and warnings. Same setup for the workflow step, except this time, write the standard output to the writebuildwarning, and error output you write to writebuilderror.



Now you setup the build definition; you will need a blank solution file... you don't need anything in it. I'm sure you can find it in the workflow where it looks for it (and remove that step), but we never bothered.

I hope this will get you started with the scripts. Let me know how it works out for you!

4 comments:

  1. Hi,
    Firstly, awesome post. Thanks so much for this
    Secondly, just wanted to ask whether you have come across a problem we are having.
    When we do the import from the combined XPO file, we are noticing that there are some objects that are not brought across. For example, some indexes are not fully created, title's not brought across, and objects missing. It seems as if it has trouble with some types of dependencies, and we are at a bit of a loss to find out why.
    We are importing a VAP layer if that helps with anything.

    Have you come across this before?

    ReplyDelete
  2. Hi Richard,
    I am wondering how you are importing that XPO. The CombineXPO script I have up on CodePlex does not do anything intelligent like sorting the objects on dependencies. The AX command-line import routing the build script uses will actually detect conflicts and re-import any objects in multiple passes. This is inefficient, but it works great. The only issue we've enounctered are views that depend on each other. The import routine checks for compile errors, and tries to re-import the objects that error out. The views however will not have compile errors, but synchronization errors, which the import does not account for. That is the only issue I know of.

    However, if you are importing this XPO manually, you will have to do the multiple-pass import yourself, or import the XPO gradually based on the dependencies you know of.

    ReplyDelete
  3. Joris,
    Thanks for the reply. We tried a number of things, and finally settled on a variation of your Combine XPO script.
    We used a CombineXPO.exe program (from http://blogs.msdn.com/b/mfp/archive/2006/11/13/building-a-layer-file-from-xpo-files.aspx) which produces a slightly difrerent XPO to the one that your script produces. It is a lot smaller (in our case, 17 MB vs 30 MB) and this seemed to work for us.
    Another gotcha we came across was having to disable UAC on the build server to get the powershell scripts to run. (They require Administrator elevation due to starting and stopping services), aswell as ensuring that the build account is a member of the local administrators group.

    Thank you so much for the scripts. They are a work of genius, and have saved me a lot of headaches.

    ReplyDelete
  4. Richard,

    thanks for the feedback. MFP and I have actually had some discussions about this. His scripts actually parses the XPOs to understand the dependencies. That is also the reason he is unable to share the code for the executable, since some of that parsing code is actually AX code, which is owned by Microsoft. I'm not sure how the file size could be that much smaller. AX 2009 XPOs use unicode, so I'm wondering if the CombineXpo executable does not use unicode?

    As for UAC, yes that is an issue, and Microsoft has been unable to give me a way to run the build service in elevated permissions, so I'm assuming there is no way at all. We have indeed also had to disable UAC there.

    As a hint, the combineXPO script we use actually also "creates" a macro on the fly, which contains the build number. That way the output layer actually contains the TFS build number for reference (and then you could put it in the about screen or something).

    ReplyDelete