Thursday, November 17, 2011

AX 2012 and TFS

During the beta program and even up to the last few months before the launch, I have talked a lot with Microsoft expressing the concerns and issues we were having with AX 2009 and TFS, and offering suggestions on what would ease some of the pains of the integration that existed. Unfortunately, not all of these issues have been resolved. Some are fundamental problems in the way things are architected, other issues are unique to AX compared to other IDEs that TFS integrates with. Luckily, there are nice improvements in AX 2012 as well.
I will give you an overview here, and post detailed information in follow-up blog posts.


The Bad (I'm one of those people who wants to hear the bad news first)

The major issue that existed in AX 2009, unfortunately still exists today: the TFS integration in AX 2012 does NOT support multiple developers in the same environment. This is a fundamental issue that unfortunately cannot be solved easily. I've had numerous discussions about this with Microsoft, and there is no good solution yet. It's part AX and part TFS that is the problem. Microsoft has come out with a whitepaper on the topic, suggesting to use public workspaces. Sad to say I had coined that idea to Microsoft back in January, but we had decided this solution would not work well. Somehow this ended up as an official whitepaper (some of the people I talked to at Microsoft were surprised about this as well). In any case, the public workspace does NOT work well. There are other workarounds (basically still the same ones I have blogged about for AX 2009) but they are not pretty. We've used them for quite some time now with AX 2009, and they have served us well without any problems. More on this later.
The other frustrating topic is models. The beauty of models is they support the object sub-level granularity in the AOT, meaning that one method can be in one layer/model, another method on the same class can be in a different layer/model, etc. This is the basic premise for models to work: you can install a model that adds a method to a class, without needing the whole class in the model. That way you can install numerous models without having to merge the code. The TFS integration however, is file-based using XPOs. The current XPO format as it has existed in AX for a while, does NOT support this granular model. You CANNOT export just one method, you have to export the main class element with it (class properties including "extends" property etc). Yes, you can have an XPO of a class with just one method, but you'll still have the class itself in the XPO. This effectively BREAKS the granular storage. Obviously this must have been a big consideration for Microsoft when creating the TFS integration. Since you include the class declaration itself in the XPO, how can you separate the different methods XPOs in different folders? Where will the class declaration reside? Everywhere? Nowhere? On top of that, you cannot take an export of just one method, the only "filter" available is to only export methods customized in your layer. No option to only export for a certain model (although I'm guessing Microsoft could easily add this). To "solve" the problem, AX will store the XPO in the folder of the model it was FIRST used in, and keep it there forever. So if you add a method to a never-before customized standard AX class and add it to model A, AX will place the XPO in the ModelA folder. If we now want to add another method to this class, in model B, AX considers the fact the XPO already exists, and ends up putting your model B customization in the XPO in the ModelA folder!! This effectively renders the model support in version control useless. Especially if you are wanting to automate builds, there is no more correlation between the model folders in TFS versus the actual models the code should belong to. I understand this as a fundamental problem that is not easily solved without changing the format and functionality of XPOs, so my only conclusion is that the model - TFS integration should not be used (or you put everything in one model). I gave this example with a class but it goes for any objects or changes, including forms, tables, event handlers, etc.

The Good

The TFS integration in AX 2012 now supports sub-folders. In previous releases, AX insisted on putting all the code starting at the root level in your source control tree. So if you want to do branching, this is a big issue. This was fairly easily remedied by changing every developer's workspace settings in TFS to point to a sub-folder in the source control tree. At Streamline we have actually automated (customized) this process in AX on a developer's first login, so it's transparent and the developer doesn't need to do anything. AX 2012 now supports entering a folder path in the TFS settings!
The check-in dialog in AX 2012 has been extended to allow you to associate your check-in with a TFS work item. This is a great feature (one which I understand came from a Microsoft developer coding this in his spare time), although I would have liked to see support for TFS queries. Currently the query for the work items is hardcoded in a Macro in the AOT. At Streamline Systems we are extending this to have a dropdown available with your TFS queries. More on this later.
The creation of the developer workspace and some other lower level TFS stuff has always been handled in assemblies. There was no way to look into or customize the behavior of how AX creates the workspaces or talks to TFS. After some discussions, Microsoft ended up putting that .NET code in the AOT as a Visual Studio project. This means if there is a need to debug or to customize the TFS proxy used by AX, you can do so in the SysVersionControlTfsFacade Visual Studio/C sharp project in the AOT.


Seems like I spent more time on the bad than the good. The main reason is I feel developers should understand the limitations of the TFS integration, and understand WHY these are big topics, and why some of those are not an easy task for Microsoft to solve. I expect we will be living with some of these issues for a while. That does not mean you cannot use the TFS integration. In fact, we use it heavily here.
In the coming blog posts, I will dive into specifics of how and why these issues exist, and offer some solutions.

In other news, we are also working on new build scripts to put on CodePlex, which will be compatible with AX 2012 and its models. Keep an eye out. For general TFS and ALM information and our solutions for AX 2009, check out the TFS page.

22 comments:

  1. Hello Joris,

    as a newbie I was able to set TFS with AX.

    It's working fine ... I only have a problem with labels ... I didn't want my labels to be under version control but I activated it.

    Now if I remove them from version control it keeps creating annoying temporary ids for labels... did u happen to have the same issue?

    Anyway, another great article on AX, congratulations!

    Ivan

    ReplyDelete
  2. Just a remark to multiple models: If you have an object with components in multiple models (e.g. with two methods in two different models), you are even not allowed to add it to TFS. The following error is thrown by SysVersionControlSystemFileBased class: "The model element XYZ has references to multiple models. This is not supported."

    ReplyDelete
  3. Hi Ivan,

    we always add our labels to TFS as well, so I don't really have an answer for you. But I do wonder why you don't add your labels to TFS? You should.

    Martin,
    I just tried and you are correct! I guess I must have tried this on an older version or even one of the pre-releases. Thanks for clarifying.

    ReplyDelete
  4. Thanks for sharing your info. I really appreciate your efforts and I will be waiting for your further write ups thanks once again.
    IPhone App Development| Android apps developer

    ReplyDelete
  5. Hi Joris.

    Have you found any work arounds for the issue of "The model element XYZ has references to multiple models. This is not supported."

    Some of our most important objects exist in multiple models. We have a main model and an integration model so that we can either ship a model on its on or with specific integration functionality to some of our other products.

    ReplyDelete
    Replies
    1. We have fixed that internally by making sure the "shared" objects are in a separate model. Then we branch per model-folder, and from the shared folder we only branch the relevant changesets... I haven't come up with any other workable way.

      Delete
    2. Hi Joris,

      Is branching the only way to get around this limitation, or have you found a better way to do this since this post? This issue is quite important for me.

      Delete
    3. We use branching anyway but yes we use it also to work around this issue. Haven't found any other way to do it. As explained in the article, the real issue is the granularity of XPOs. XPOs are what they are, until they are changed in some way, this issue will exist.

      Delete
  6. Hi Joris

    I can't get public workspaces to work with AX 2012 R2.
    It works as normal for the owner of the workspace, but everyone else can check out items (as the owner) but can't check them back in. The checkout option is still visible in the AOT context menu.

    Do you have any hints to resolve this issue?

    Regards Brian

    ReplyDelete
    Replies
    1. We don't use public workspaces so besides some testing I've done I don't really have much experience with that. Did you go through the whitepaper Microsoft published on the subject?

      Delete
    2. Yes I did.

      We seem only to have this problem om Windows 2012 server environments though. We also have Windows 2008 R2 servers that connect to the same TFS server without these problems.

      Brian

      Delete
    3. I'm afraid I don't know. I suggest asking this question on a TFS forum, try on http://social.microsoft.com/Forums/en-US/categories under Visual Studio or try on StackOverflow.com or something. I'm guessing this won't be AX-specific but rather a TFS issue.

      Delete
  7. Hi,
    Is this statement still true?

    "The current XPO format as it has existed in AX for a while, does NOT support this granular model. You CANNOT export just one method, you have to export the main class element with it (class properties including "extends" property etc). Yes, you can have an XPO of a class with just one method, but you'll still have the class itself in the XPO"
    If so then do I have to manually move a class that is split between models into a model so that all methods etc are in a single model?

    ReplyDelete
    Replies
    1. Just done a bit more testing and need to get clarification, if a class has methods in two models but the models are in different layers, can TFS cope with this ok? It seems to but I wanted someone with more experience than me to look at this too.

      Delete
    2. It's not an issue to have multiple models or layers controlled in TFS, but what you do on the AOS side may be the issue. Yes it is supported to have a model in one layer and a model in another layer both controlled into TFS from the same AOS, but it does present a few challenges. I haven't tried this in 2012 but tried it briefly in 2009 and there were some serious drawbacks. I would consider rethinking why you need it setup that way and whether you can do with just 1 model in 1 layer, or whether you can make different environments (AOSes) to develop each in separately.

      Delete
    3. Hi Matthew,

      I actually tried this as a solution in 2012 R2. I found it does keep it granular when checking into TFS. However, when synchronizing it would take the class from one or the other model, but wouldn't pull the methods from both. I didn't spend much time on it but it definitely doesn't look straightforward.

      Joris, correct me if I'm wrong, but the underlying problem isn't so much with the XPO format itself, but the fact that you can only export a whole layer into the XPO format rather than individual models. If the models are in different layers, the XPO output will be granular. I noticed that if you manually export a class into XPO, you can select an option to specify a single application layer but there is no option for a single model. Perhaps if this export option is added in a future version, this multiple model limitation can also be resolved.

      Delete
    4. You are correct. If we had a feature like "export from layer" called "export from model", that would probably fix it.

      Delete
  8. Hi, We are looking got go to a single model. We split is to separate projects done by offsite resources at their request not ours. It has been painful

    ReplyDelete
  9. I do have a question about shared AOS topology though. We have one AOS, three developers who develop on the one AOS via local clients. One who logs onto the AOS server via RDP. The three developers who use local clients use local private workspaces and cannot access the shared workspace that I have created on the AOS server for the fourth developer. Is this correct behaviour or could I have done something wrong?

    ReplyDelete
    Replies
    1. I don't think there's an issue with that setup.

      Delete
  10. Get the following error when trying to sync in AX, anyone know what the cause/solution to this is?

    "Cannot create a record in Version control changes (SysVersionControlTmpChange). "...

    ReplyDelete
    Replies
    1. missed the last bit... "the record already exists"

      Delete