Tuesday, November 5, 2013

What You Need To Know About the 15-minute CU7 Compiler

If you hadn't heard yet, CU7 has been released for AX 2012 R2. Lots of things to get excited about (comparing reports anyone?), the thing I'm most excited about is the new compile optimization. We did a lot of work in the past to identify the bottlenecks and speed up the compile by minimizing those bottlenecks. If you're on AX 2012 RTM, that's your best bet. However, the main issue still remained: the compiler didn't use more than 1 core and so didn't scale with hardware.

So, no more! CU7 includes a new way to compile the X++ code (CIL is still the way it was). First, it eliminates the client altogether, which removes another communication bottleneck by taking the 32 bit client (and the communication with the AOS) entirely out of the picture. The other advantage to this is that the server component is a 64 bit process, allowing access to more memory. Secondly, it will spin up multiple processes to compile pieces of the X++ code, effectively using multiple cores.

There are a few gotchas that you need to be aware of, though.

By moving the compile into a 64-bit process, there are limitations to loading certain dependent client-side (32 bit) components for the compile. These could be ActiveX controls, COM objects or even DLLs and assemblies. To be able to do the compile, the compiler will use reflection on all these objects, effectively allowing it to check everything without having to actually load these components.
Obviously to be able to check all these dependencies correctly, the compiler needs access to client-side components it typically does not have installed. To that end the compiler needs the path to a folder that contains all these dlls it may need. Typically pointing it to the folder of an installed AX client is enough.
When calling COM objects in X++, you can "chain" the method calls (for example: COM.method1().method2(); ) relying on the fact that the first method call will return an object of a certain type on which you can call the second method. With the server-side compile, this chaining cannot be checked, so code has to be refactored into multiple calls (for example: o = COM.method1(); o.method2(); ). Note that you will run into this trying to compile a model store version prior to CU7 with the new compiler (SysInetHTMLEditor has calls like this that need to be fixed).
Finally, because it starts multiple instances of the server process to do the compile, hotswapping needs to be turned OFF.

So, how does it work?

Well, first: check and make sure hotswapping is turned OFF! Next, open an elevated command prompt ("run as administrator"). Next, navigate to the server/bin folder which contains the axbuild.exe program. There are several parameters you need, one optional one controls the number of parallel processes it uses (aka "workers"). By default axbuild will create roughly 1.4 workers per core you have: I have 4 cores in my laptop (2 physical, total 4 virtual), so axbuild will start up 6 worker processes. I found that reducing that to 4 workers on my machine gained me about 10 minutes. So play with this a bit to find the optimal setting. Additionally you need to specify the AOS you are compiling for. This uses the AOS "instance number" on the machine you are on. On my laptop I'm testing this on my primary AOS which is "01" (yes you need the leading zero). Finally, as mentioned earlier, you need to provide the path to a binary folder containing client-side DLLs, so pointing it to the client/bin folder works great.
My final command ends up being:

cd \Program Files\Microsoft Dynamics AX\60\Server\<YOUR AOS NAME>\bin
axbuild.exe xppcompileall /aos=01 /altbin="C:\Program Files (x86)\Microsoft Dynamics AX\60\Client\Bin" /workers=4

I'm currently running a laptop with 3GHZ i7 (2 physical cores / 4 virtual) with 16GB memory and an SSD drive, running AOS and SQL on this machine (and SQL constrained to 4GB memory). I consistently hit 15 minutes compile time.


As mentioned earlier, if you are planning to use this on CU6 or earlier, you will have to cleanup some COM calls. You will see these as compile errors in your compile log output, which will be located in C:\Program Files\Microsoft Dynamics AX\60\Server\<YOUR AOS NAME>\Log (you can specify an alternative path in the command line).

For more details on command line arguments and architecture, check out these two Microsoft links:
AX Tools Blog about CU7 Parallel compiler
MSDN Article on AXBuild


18 comments:

  1. Good post, Joris!

    One more thing that might be of interest for the techies; those temporary compilation AOSes will run under the developer/users account and not the AOS Service Account. This mean the user initiating these compilations needs to have the same permissions on the models store database as the AOS Service Account. I guess that is not a problem in most cases. :-)

    ReplyDelete
  2. Good work, and thx. I hope they will extend the AXbuild With more parameters, like CreateXref :-)

    ReplyDelete
  3. Hi Joris,

    really nice article. I have one more question.
    You mentioned that using an application prior to CU7 needs some adjustments in the application code.
    Does this mean that having an installation with a prior version than CU7, doing a binary update only to CU7 will provide the AXBuild.exe feature?
    We have many AX2012 versions to maintain but the are on different version (AX2012 FP, AX2012 R2 RTM, AX2012 R2 CU6). Doing an application upgrade is not possible at the moment for all of them but a binary update is less time consuming.
    Many thanks in advance for feedback.
    Thomas

    ReplyDelete
    Replies
    1. I have not tested with versions before R2 CU6, but I expect it would work as well. You do need R2 though, you can't try it on RTM. But yes, updating the binaries will give you the axbuild process.
      One of our developers also found that you need to commit to this kernel. You can't just use CU7 for doing compile and then actually run the code using a kernel prior to CU7. The compile starts the AOS service and it will make changes to your database that make it incompatible to run again with a prior version (without SQL data hacks anyway).

      Delete
    2. Thanks for feedback. A kernel upgrade will be a suitable solution. At least we can afterwards use all processors of the machine and reduce the build times.

      Delete
  4. Update: the command line that was showing was missing the "xppcompileall" argument, I just updated that. Thanks to our friends at Microsoft for pointing that out :-)

    ReplyDelete
  5. I am trying with CU1. I have Cu1 installation for which the kernel is upgraded to Cu7 to get the build tools .... but when compiling in pass2 it crashes with following error: http://tinypic.com/r/68dkhv/8

    any suggestion?

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
    2. This comment has been removed by the author.

      Delete
  6. Hey Dick de Jong, Noticed that you have assigned 12 workers in your command referring to the screen shot (http://tinypic.com/r/68dkhv/8) which may be wrong, do not specify the workers command and let the system manage it. Make sure the AOS is stopped while executing this AXBUILD.exe server side tool.

    ReplyDelete
  7. Hi,

    I would like to know why did you say that hotswapping must be turned off (in capitals, with exclamation mark) :)

    Thanks

    Nicolas

    ReplyDelete
    Replies
    1. Could be fixed in hotfixes or R3, but it caused a bunch of errors with the AOSes trying to compile. I couldn't tell you why, but it did. Again - this could be fixed in newer versions, this article is almost a year old by now, and our build machines have it turned off anyway so I haven't actually tried.

      Delete
    2. OK thanks. I was asking you because just before I read the article, I did a R2 axbuild compilation with this option checked...and it went fine. So I was wondering if I wasn't sitting on a time bomb ready to explode ^^
      You must be right, it may have been fixed in hotfixes.

      Delete
  8. Hello I have a simple question.
    When I run the axbuild command, should I stop AOS service or leave AOS service running

    ReplyDelete