Thursday, July 21, 2011

AX 2012 RecIds and UnitOfWork Foreign-Key Magic

AX 2012 features the new UnitOfWork class. One of the reasons this class exists, is to handle new issues that have come up because of one major shift in thinking in AX.

We now should try to link records together based on RecIds.

I can hear you scream WHAT all the way to Denver. Yes, you heard me right, it's now considered GOOD practice to link records together based on RecIds. Most developers I know that have any kind of experience with AX hate tables that link on RecId (although in some cases you have to), and usually consider it bad practice.

Well, get with the program! :-) It is now considered a good practice. Key reasons are performance and avoiding data duplication. Lots of tables in AX use(d) long strings as primary keys, and in some cases you need to start linking on more than one field to keep linking unique, etc. Well, since RecId is always unique within a table, and it's "just" a number, the duplication of multiple field values, and the performance on long string primary keys is now "solved".
Yes, I agree, this will not be pretty when doing data migrations and company copies (which for all intents and purposes are near impossible in 2012 - more on that later). But I'm sure somehow we will all get over our preconceptions and embrace this new paradigm (I'm using it, but I'm not a big fan just yet).

In any case, it is what it is. There are some new things around this RecId linking, to facilitate both developer as well as end-user (can you imagine a user having to pick a RecId from a dropdown instead of a Customer number? Before you freak out - customers still have account numbers and that's still the primary key).

Let's create a new table called AX2012Features, which uses RecId as its primary key, and a Name field. This name field will be contained in a unique index as well, and be designated as the "Alternate Key" (more on this later). On the table's properties, set this key as the "ReplacementKey".

Next, we create a new extended data type for our RecId (yes!). So this data type we'll call AX2012FeatureRecId, and it has to be an Int64. As it so happens, this data type has a new property called "ReferenceTable", where we fill in our AX2012Features table.

I'm skipping the creation of a form for this table. Well just add some records using the table browser, we're amongst developers here, right?
So, on to the next table. We'll add an AX2012FavoriteFeatures table, where we can add an integer field "Ranking" for our favorites features from the other table. Also, we will drag and drop our new AX2012FeatureRecId extended data type on the "Fields" node of the table, which will then release some magic. I will rename this field to "FeatureRecId".

When you now open this table's table browser, nothing appears out of the ordinary (besides the fact the table browser in 2012 is now finally no longer an MDI child). There is an integer ranking field, and a FeatureRecId field with a dropdown with recids.
Let's create a form for this, shall we? Now, for the fields, either drag them from the datasource onto the design, OR, when you add a control for the FeatureRecId, select the "ReferenceGroup" control instead and set the referencefield property to "FeatureRecId". If you drag & drop, it will take referencegroup automatically.

A picture says more than a thousand words, so open up the form, which now has the FeatureRecId field (an Int64) in a ReferenceGroup control.

Yes indeed. The dropdown features your alternate key with your name. If you select a value, the supposed RecId field actually shows the Name, not the RecId. Please try this out with more than one field in the AlternateKey, instead of just one like we did here.
Ok, so this is pretty cool actually. We only store one RecId field, but the user sees and picks from a list of values from the alternatekey (however many fields there may be in that key). This begs the question, what do we do in code? Traditionally, I know going in, what my name will be, and so I can add that name when I insert a record in the favorites. Now with this, I cannot get my inserts going, until I have added the favorites, then I need to get that recid and put it in the favorite.

Blast! Just when you started getting used to using RecordInsertList, you need to stop using it because you need the auto-generated value to insert another record. Well, have no fear, the UnitOfWork class is here. And even better, if you have complex structures, it will figure out the sequencing of inserts for you! How you say? Let's create a class, sporting our two tables and a UnitOfWork class. We create an instance of the UnitOfWork class, and we set the name of our empty feature to "UnitOfWork". Instead of calling insert, we call unitOfWork's "InsertOnSaveChanges", to indicate we want to insert this record when we call "savechanges" on unitofwork. We use a class to test this, and not a job, because UnitOfWork always needs to run on server. So, when you create a main method to run the class, make sure to declare it SERVER!

Next, the magic happens. Go back to your FavoriteFeatures table, and check the relations. The RecId data type prompted you to auto-create the relation. If you open the relation, you'll see the FeatureRecId field linked to the RecId of the Features table. Open the properties of the relation and set the "CreateNavigationPropertyMethods" to YES.

In the code, we'll say our number one feature (at least for this blog post)is the one we're trying to insert. Because of the ForeignKey relationship, and setting the navigation property methods to YES, AX 2012 has added a method to the favorites table, carrying the name of your relationship, in the case of this example, AX2012Features. We pass in the (not yet created!) record buffer of feature. Next, we add the favorites buffer to the unitOfWork as well.

Alright. So now, the unit of work class knows we want to insert both records, and it also knows which one depends on which other one, so it will figure out in what sequence the inserts need to happen (granted, in this example there's not much to it… but feel free to experiment with more records, or changing up the order of calling "insertonsavechanges").

Since UnitOfWork also takes care of transactions, we don't really need to do anything more than call the SaveChanges method (because of this transaction model, if you want to use savechanges to make updates, make sure to select your records for optimistic concurrency!).

So, your full code should now look like this:

So… ready to run it? GO! And yes, if you open the favorites form, you'll see your unitofwork ranking number 1 in there, properly linked on RecId (but of course sporting the alternatekey "Name" field on the form instead!).

Wednesday, July 20, 2011

AX 2012 Models & Events Part3: The Power Combo!

In the conclusion of this 3-part blog article, I will finally get to my point. We will walk through the event code and models and create an example where we have two models that both touch the same table, the same form AND the same method (by means of handlers). In the end, we will import both those and they will install themselves, without any merging necessary. They can be used independently, or together. Hopefully you will understand the needed change in thinking and designing code to make this possible.

Part 1: Model Architecture
Part 2: Events
Part 3: The Power Combo

So, let's get down to business. We will create two models (separately, one at a time), each will add a new field to the CustTable table, each will add that field to the customer details screen, and each will add code to the validatefield method on the custtable. We will create one model, do the customization, remove the model, create the other model, do the customization, then import the old model back in. If the promise of models and events holds, and if we create our customizations the right way, we can add and remove these models in whatever sequence we want, as many times as we want, and there will be no merging necessary at all. Just a plain old install, as if you would install an app on your phone.

If you are attempting this walkthrough on the public beta, there are potential issues. Remember that is a pre-release which is 5~6 months older than the version I have created this exercise on. If there are issues with the beta, please let me know in the comments. And be patient and try again when RTM is released (we're getting close!).

Each layer in AX 2012 has a default model. For VAR, this would be the "VAR Model", CUS is the "CUS Model", etc. So first thing we need to do, regardless of what layer you are test this in, is create a new model so we can separate our code. Open AX in the developer workspace (add "-development" in your client configuration or use shortcut CTRL+SHIFT+W in the regular mode to open the developer workspace). From the Tools menu, select Model Management / Create Model.

Enter the information of the model. You don't have to copy my settings exactly, just make it unique enough, our next model will need a different name (and remember what is what). I decided to name this first model DAXMusings after my blog, and the publisher is myself. Make sure to give it a display name as well, and set it as your current model. As opposed to layers, you can change "active" models on the fly (only within the layer you're logged into, though).

You will get a nice infolog message indicating your model was created successfully. Other ways to create a model is from the command line using the AXUTIL command-line tool. If the tool is not in your path, it's located on your server, in the 64-bit program files under "management utilities". Remember though this is a separate item in the AX 2012 installer.
Next up, we'll create a new project and drag the CustTable table and CustTable form into it. We will also create a new class, which, to keep it easy, I will also call DAXMusings.

Ok, so here's the story. We will add a field on the custtable to contain a project number (string extended data type "ProjId"). We will validate that field to make sure the project status is "In Process" (granted, this is a silly, useless example, but for the sake of the exercise, let's pretend it's business critical!). Traditionally, we would add code to the validateField method on the CustTable. This would be a one-liner calling a method somewhere else, or we would just add all the code in there, doesn't matter. However, in AX 2012, we can segregate our customization easily by using a pre/post handler. In that case, come the next hotfix rollup or AX release, we need not be concerned about "upgrading" our over-layered method, because it won't be over-layered at all. And, as we will see here, when we install another model that also needs to hook into the validateField, we can just install it without investigating both code sets and merging.
So, let's get down to it. First, we add our field. To avoid more conflict, we will prefix our customization objects with DM (for DAX Musings). Add the field DMProjId, and set the extended data type to "ProjId".

We will add this field on the CustTable form, on the general tab. It doesn't really matter where you put it, the point is we're customizing the screen by adding it.

Next, we will add a pre/post handler method to attach to the CustTable.validateField method. We will use the standard pre/post handler, since that allows us to change the return value. In case of the validateField, changing the return value makes sense. By default, it will return true, and if one of the validations fail it will set the return value to false. So, if we (and consequent other handlers) don't touch the return value unless we want to set it to false, it doesn't matter what order handlers subscribe (let that sink in - I'm making sense, really). I will spare you the details on the code I'm writing. You can just either set the return to false or true to speed this up, I'm still trying to maintain the illusion of a useful mod here.

To hook up this method, drag&drop the method onto the validateField method of the CustTable, or right-click the validateField method and select "New Event Handler Subscription". Make SURE to rename the subscription to a unique name, and set the "CalledWhen" property to "Post" (if you select pre, the return value will be reset to the default TRUE of the original validatefield method!).

Feel free to open the CustTable form and test the validation. Trust me, it works! :-)
Great, so now we have our DAXMusings model. Let's export this to an AxModel file, and remove it from our system. To do this, open up a command prompt or powershell (I prefer the latter after discovering the joys of powershell). We will use AXUTIL to export, and then remove the model.

The tool will ask you to restart the AOS (think: indexing of objects/layers). After you restart the AOS and fire up your AX client again, you will get a prompt asking you to synchronize and compile. Obviously, you should. In our case, the table needs to drop a field, and the validateField method of the CustTable table needs to be recompiled (remember the pre/post handler is compiled as if it were part of the original method, so a recompile is definitely needed here - weird and bad things happen otherwise).

So, now that we're back to a virgin AX with foundation, let's repeat the steps. This time, we'll sneak in some clandestine advertising for Streamline Systems in our model name and description. As a field, we'll add an item number (string extended data type "ItemId") and validate that it does not have an alternate item number (field AltItemId should be blank on the InventTable). Otherwise, we're pretty much repeating steps here (fast forward the screenshots!).

Add a field
Set the Extended Data Type to "ItemId"
Create pre/post handler method on the new class
Subscribe the new method to the validateField on CustTable
Add the new field to the CustTable form

Alright. Now that that's over, think about what we are about to do. We have an installed model customizing the CustTable, adding a field to a form, and performing a validation in code by adding code, but by using pre/post handlers rather than actually modifying the base code.
So, let's put this new architecture to the test and install our old DAXMusings model back into this same environment.

Now, restart the service, and let's fire up AX. Again, it will prompt about model changes. Synchronize and compile people (seriously, compiling is an absolute must here)! So now, before you test the mod, what do you think will happen?

YES! Success! (admittedly, I was sweating a little bit… all this typing, imagine if this hadn't worked).

Open the CustTable form… Both the fields are there!

Try to pick an item with an alternative item (3004 in demo data). Try to put in a project not in process. (if you are getting unexpected results… did you compile?)

Enjoy the power of this new architecture. If you have doubts, feel free to import/export these models in other environments that haven't been customized at all. You can also try to ACTUALLY customize the validateField method, you will notice it is independent of the handlers attached to it (proving the upgrade point, if standard AX modifies lower level code).

Hope you enjoyed the series. Please leave me a comment, or send me some feedback using the contact form!

Tuesday, July 19, 2011

AX 2012 Models & Events Part2: Events

In this 3-part series I will let you in on the power of the 2012 model architecture in combination with events. There is a paradigm shift in regards to customizations and even ISV code that will make life so much easier, provided everyone uses this new model. To that end, I will talk a bit about models, events, and then put it all together and SHOW you the advantage. The originally article kept getting bigger and bigger, so I decided to split it up in three parts, I will publish one every day.

Part 1: Model Architecture
Part 2: Events
Part 3: Full Walkthrough (with code and screenshots)

AX 2012 introduces the concept of events (whitepaper is available on Microsoft's Download Center - for a list of all available 2012 documentation see my June Documentation round-up post). For those of you familiar with C# you will realize that delegates are a great concept, but they require the base functionality to have these delegates (for those unfamiliar, delegates are event "hooks" the application exposes). I counted the number of delegates in the RTM release of AX 2012, and there are about 100 (which is not a lot yet, considering the application has around 5 million lines of code). Chances are your customizations will want to subscribe to places, or make modifications to, code that doesn't provide any delegates. Peter Villadsen and his team thought about this, and introduced a concept of pre and post-handlers. This basically allows you to "subscribe" your code to any method in the application, without having to modify the standard application code to make that call. For the sake of customizations, I will talk about these pre/post handlers, and not delegates. The principles are similar enough. I will have another article on delegates altogether, since there are some interesting details to them.
In any case, pre-post handlers come in two flavors: the "lightweight" handlers, and the other handlers that don't have an official name (yet?). Let's call them original handlers (one could argue they should use some opposite term of lightweight, but let's not go there).
Let's start with the original handlers. They are always static methods (since pre-post handlers can only be attached in the AOT, as opposed to delegate handlers that can be subscribed at runtime), obviously public (remember, new methods in AX 2012 will be explicitly declared "private" by default), and no return type (=void), and accept an object of type XppPrePostArgs.

The XppPrePostArgs object is the juice of the meat. This gives you access to the arguments passed into the method you're subscribing to (using getArg() or args()), the return value being returned from the method you're subscribing to (getReturnValue()), and it has a method getThis() which returns the instance of the object that fired the event. Of course, you can modify the arguments being passed in, and modify the return value of the original method. But here's the thing! Technically, it is not guaranteed in what order the event subscribers will be called. This is because it depends on the order in which code is imported, models are installed, etc. So changing the return value (in post-handlers) or arguments (in pre-handlers) is possible, but since you are not guaranteed to be the last person to touch these values, it is not guaranteed that nobody will overwrite your values.
Since changing the return value cannot be guaranteed, it becomes a little less useful. On top of that, accessing arguments through the args() or getArg() methods is not the safest thing. You either access the arguments through index or through name (ugh), which you won't know works until runtime. Because of these limitations and potential issues, lightweight handlers were invented. They are lightweight in the sense that they do not provide a means to change the arguments (but you can read them - and remember certain types are passed by reference so technically you can change them), no means to change the return value, and no means to retrieve the object instance the event was thrown from. Unfortunately, the latter makes the lightweight handlers handicapped to a certain extent. All in all, depending on your situation, you will find that either one will support you in your endeavors. It's a powerful feature.
To finish the quick introduction (without spoiling the next article), to hook up the subscriber, you either drag your subscriber method and drop it on the method you wish to subscribe to. Or you right-click the method you wish to subscribe to and select "New Event Handler Subscription".

Note that the "subscription" itself (the node in the AOT) carries a name property. Make sure to give this a unique name. Also in the properties, you can denote the subscription to be pre or post, and in case you wish to write the code in managed code, you can, by setting the EventHandlerType property to "Managed". In that case, make sure, for the class name, to include the full assembly name as well (eg: MyCompany.MyProject.MyClass).

There are two major things you need to know to appreciate this pre/post handler feature even more:
  1) The "subscription" (the link between the original method and your subscriber method) has an AOT node, and is stored in the layer/model separate from the original code (so giving it a unique name is of the essence!)
  2) The compilation of the original method will "append" the call to your handler to itself - this means no "lookup" overhead at runtime (if you think about this, it will answer any questions you may have around database transactions, exceptions, etc… consider the pre/post handler as if they were calls to your method at the very beginning, or very end, of the method you're subscribing to).

The key here is number 1 though, the fact that the subscription is stored separate from the original method. This is in fact the number one reason for the power of the events and models combo!

Next: Part 3: Full Walkthrough (with code and screenshots)

Monday, July 18, 2011

AX 2012 Models & Events Part1: Model Architecture

In this 3-part series I will let you in on the power of the 2012 model architecture in combination with events. There is a paradigm shift in regards to customizations and even ISV code that will make life so much easier, provided everyone uses this new model. To that end, I will talk a bit about models, events, and then put it all together and SHOW you the advantage. The originally article kept getting bigger and bigger, so I decided to split it up in three parts, I will publish one every day.

Part 1: Model Architecture
Part 2: Events
Part 3: Full Walkthrough (with code and screenshots)

In this first article, we need to learn some basics about models. The standard layering system as we've known it for years still exists. This means, when you modify an existing object, it gets copied into the layer you are working on, and your changes will be saved there. This means the existing object is still intact in its original lower layer, but AX uses its index to execute the object in the highest layer it can be found in. There are however some interesting changes in AX 2012 even on that front. Certain objects such as reports (which only exist in AX 2012 for backwards compatibility) and forms, used to get copied into your working layer, in their entirety. This meant, even if you just added a method or control to a form, or move a control to a different tab, the whole form was over-layered. This opposed to classes, where adding or changing a method only puts THAT particular method in the higher layer, not the whole class with all of its methods. This has been changed in AX 2012. Forms are now over-layered on a control and method level. This is achieved by giving each control a property that indicates what parent it is attached to. This means AX can now keep track of each control. If you move it, the control still exists, it just changed its parent property. So when you add a new control, it is the new control that gets put in the higher layer. For customizations, this is already a blessing, making it much easier to merge customizations, merge a customization with a hotfix rollup, etc. This also means in the AOT you can now see your traditional tree-view design, or you can look at the flat list of controls. This may be necessary in situations where controls get "orphaned" (the parent control is removed). There are some strategies when importing to deal with these situations.

So, back to the models. They are basically compartments within each layer. An object exists in one or more layers, and in EXACTLY one model per layer. Let that sink in. The immediate question then becomes: what good does it do if an object can only exist in one model per layer? Well, there's a few things. For one, since we no longer have AOD layer files to move, we can export models from a layer to a binary format, and move (export/import) those between environments. This has the added benefit we don't have to move the whole layer, but logically segregated pieces of code, such as one custom module at a time, or certain third-party apps that are installed in the same layer. Models have an extension of .axmodel, and can be signed (like assemblies) to ensure integrity and authenticity when distributing. The "import" (aka installation) of a model can be done using powershell or regular command line, or you can create a custom installer package to install the model. Of course models can be "uninstalled" in the same fashion. All that is required is a restart of the AOS (think rebuilding the index in previous versions). When you then open AX, it will prompt you with a warning and some choices related to compiling, synchronizing, etc.

I can see you wonder (yes, this blog enables your webcam so I can see you wondering) how this is going to work. Sure, you say, forms are now granular to a control-level, but still… when it comes to code, multiple mods may be touching the same piece of code. This will require merging on install, and it will mean the object/method is one model or the other, which makes model moves useless, since I'll have to move them all, or resume my old-school XPO practices and merge on import.

AHA! You hit it now. Thank you for leading me to the point of this article series. Events provide the missing piece to the puzzle.

Next: Part 2: Events

Wednesday, July 6, 2011

Generic Unit Tests for Dynamics AX in Visual Studio

A little while ago, I blogged about unit testing and integrating this with TFS builds. We have this running currently, but with some drawbacks. We basically wrap each X++ unit test method in a test method in C#, which reads the AX unit test XML results file and either throws an error or passes the test. As Dave Froslie pointed out to me, there is such a thing as "generic tests" in Visual Studio. This is basically a means to invoke an external process, which can then generate a summary results file in .TRX (xml) format.
That of course would be a much more efficient way of doing this. Seeing that AX has "listener" classes, which can subscribe to the unit test framework's output, all we need to do is create one that writes the TRX format that Visual Studio is expecting…

I have created the listener and it's up for download on the AX Build project on CodePlex. In this post I will describe how to set up the generic test in Visual Studio 2010. This process works for both AX 2009 and AX 2012. The code is the same, but make sure you download the right XPO from CodePlex (AX 2009 doesn't like the AX 2012 XPO format).

For a quick overview on how to create unit tests in AX, please check out the following pages on MSDN for AX2009 and AX2012.

For the purpose of this walkthrough, we'll assume you have an X++ test project containing your tests (aka "test suite"). Let's also assume this project is called "TFSGenericTestProject". Replace with the name of your actual unit test project in AX.

First, download the SysTestListenerTRX class for AX2009 or AX2012 from the AX Build CodePlex project. Import the class into AX. This implements the TRX format for a unit test "listener". A listener is a class AX unit tests use to output their results. TRX is the result file format which generic unit tests in Visual Studio can read. Feel free to open these, they're basically XML files. To make the generic unit tests work, all you need is the class. The enum you'll need if you want to run the TRX listener when running unit tests inside AX, but is not necessary for the TRX listener to work.

AX unit tests can be run from a command line. You can start the AX client as usual (either using the default config in the registry, or by passing a config file as the first command line argument. Furthermore, you can pass "RunTestProject" as a startup command, followed by an underscore, followed by the name of the AX unit test project. The syntax for this is a little strange perhaps:


Where "TFSGenericTestProject" is the name of your test project in AX. Now, the parsing of these arguments get even more interesting. The class that runs the unit test project supports two more arguments; one that supplies the name of the listener, and any subsequent arguments will be passed into the NEW method of your listener (the TRX listener's NEW method accepts an optional filename). The listeners are all called SysTestListener_NAME, and it's that "name" you need to pass as an argument. So, putting this all together, to start AX using a configfile.axc, run our TFSGenericTestProject with the TRX listener, and output the file to c:\unittest.trx, you would use the following command line:

Ax32.exe "configfile.axc" -StartupCmd=RunTestProject_TFSGenericTestProject@TRX@C:\unittest.trx

Now, if your filename/path contains spaces, you will need to wrap the whole thing in quotes. Since the -StartupCmd and the consequent class names etc are considered 1 argument, you would have to wrap it from start to end, as such:

Ax32.exe "configfile.axc" "-StartupCmd=RunTestProject_TFSGenericTestProject@TRX@C:\my folder with spaces\unittest.trx"

Ok, so now we're ready to create our Visual Studio generic test project. If you already have a solution (.sln) for your TFS project (which you will if you're using builds), you will add a new project to the existing solution, don't start a new solution. For the new project, select the Test template / Test Project. This will actually create a new test class for you, but we do not want to use this, so go ahead and delete that UnitTest1.cs file. Right-click your project again, and select Add / Generic Test.

As the existing program we wish to wrap, enter the path to your AX client (Ax32.exe). The command-line arguments are the arguments we talked about earlier, except one difference. The generic test supplies a variable for the test output directory, which we can leverage for our summary results TRX file. The only thing we want to change is the result settings, where we can specify the filename/path of the TRX file. Also note there is a time-out feature (this can be useful in case your ax client can't connect to the AOS).

Once you have this setup, you're DONE! Assuming you actually have an AX unit test project called "TFSGenericTestProject", you can now run this test, and you will see it open AX, run the tests, close AX, and Visual Studio will show you the results of your tests. Double-click on the test results at the bottom to see a break-down of your methods, with their results and the messages associated with them.

The test project should be automatically included in your TFS builds if you're using that, assuming you added the test project to the build solution, and it contains "test" in its name. In your build definition under "Process" you will find a section under Basic that runs the tests based on names containing *test*.