Let's compile the thing!

By Jakob Steen Hansen, Development Manager, Dynamics AX Developer Tools and Business Intelligence.

One thing that can only be underestimated is the value of having a heartbeat for your product. The heartbeat provides a regular rhythm to your engineering process, ensuring that the state of the product is well-known on an ongoing basis. A heartbeat consists of many different parts, the build process being one of the most prominent ones. When working with AX, how are you going to get this heartbeat? It all depends on how advanced your engineering practices are.

In this blog post and some to follow, I’ll talk about the purpose of a heartbeat and show how to establishing various levels of automation, including a nightly build and tools/techniques we can use to optimize the engineering system when we have a nightly build. My examples are going to be focused on AX 2012, but many of them apply to AX 2009 with little or no change.

Let’s look into the specifics for the two extremes of engineering systems I see used:

1. Opportunistic shared one box development – one or more developers share AOS and database. What is on this shared machine defines the product at any given time.
2. True distributed development – external version control system as your universal truth and every developer has their own local setup.

For (1), the basic heartbeat is making sure that what you have is in the right shape. When I say shape in this context, it’s typically fundamentals like “does it compile?” “Are we consistent with the best practices?” Sometimes it extends into “do our unit tests pass?” Multiple people working real time on a shared server will leave us with a frequent consistency issue. A heartbeat for such a system will likely have to be timed to the project schedule as the system will be in a more or less broken state for parts of the development as someone will always be working on something new and unfinished. The value of a daily run depends largely on the ability the team has to filter noise from true failures. We’re going to look at the individual tools necessary to orchestrate the automation, but we cannot generally solve for the ad hoc nature of this kind of concurrent engineering.

For (2) it’s a somewhat more formal process, ensuring that what we have under version control will actually turn into a product when we put it all together. It is also a process that has the potential to deliver output to streamline the daily system setup for development and test.

There are no specific requirements around the frequency of the heartbeat – as with many things in life it’s about balance. Don’t go overboard with too high a frequency as you will probably spend too much time maintaining a system that will not provide enough value back iteration by iteration. Don’t make frequency too low either as issues will accumulate (snowball) and you will only realize when you’re already in too deep. A daily heartbeat – typically referred to as “the nightly build” - can satisfy most.

Before venturing into the details, I’d like to run down a practice that I consider non-negotiable for a successful heartbeat:

The product builds cleanly. While we can have large arguments about whether a certain class really needs to compile when the customer is not using that functionality, it’s just a dead end conversation. It is only going to be more costly in the future if you have
broken code sitting around. If you have part of the code base you don’t want to deal with functionally, you should do what it takes to make it compile and make sure it throws an exception if ever called anyway. Go ahead and comment out offending code you don’t want to fix up, but make sure it compiles and make sure that even if someone else changes the system to call this code (trust me – happens too often), it will fail visibly and with good diagnostics. This includes all test assets in the product by the way, so since you’re adding unit tests (wink-wink), these need to compile cleanly too!

Then there’s a set of practices, which I favor, but where your individual approach will have to weigh in:

Unit tests execute successfully. At least they need to execute and when you’re really at it, they should be successful.

Code is well written and complete. Ensure that best practice tools are leveraged to maintain high code quality. With a well-established heartbeat you might even introduce a set of rules that you apply after the fact to increase your due diligence, but where you don’t want to burden every developer with them as they do the change. These baby-sitter tasks are a great source of quality improvements when you have an automated engineering system.

Enough said - now let's get to it! The first level of automation we’ll look into is the compile. This is helpful regardless whether you’re running on a shared environment or on your own. The main purpose is to ensure that we know the syntactical state of the application. Typically, this is what you do by choosing compile from the AOT root node, but for automation purposes, this process can be executed from the command line using the following command:

ax32.exe -startupcmd=compileall -development

By adding -development, we save a few cycles otherwise used to setup the application workspace, main menu etc. In the big picture it’s not much of a difference, but no reason not to take it. As the compile process is extensive and takes a good while to complete, we want to cut all the excess we can. Ensuring that you’ve disabled VCS (if using a file based one) is another way of saving some time during compile. At some point, I'll post about what can be done with respect to the hardware and configuration to generally optimize the execution. To give a sense of what’s to expect, a full compile with warning level 3 set in development options and an application that is about 20% larger than SYS alone, takes around 4 hours to complete on a Hyper-V VM  configured with 2 cores/6GB/one virtual disk on an aging 2.3GHz XEON workstation with 7200 rpm SATA-3 disks. This is not a very sophisticated setup (no RAID, no ultra-fast disks, tons of memory, etc.), so there’s quite some headroom for improving the times. As comparison, a top of the line server can accomplish this in less than 40 minutes due to its efficient I/O system. But unfortunately that’s not the box under my desk…

When the process completes, it will have written a log file with the results of the compilation. The log file can be found here:

"%userprofile%\Microsoft\Dynamics AX\Log\AxCompileAll.html"

The log file will list errors, warnings and best practice deviations. By creating a simple cmd file with the command, it can be executed at given scheduled intervals. There are many obvious additions to this basic script. As an example, you could copy the log file to some archive directory adding the current date to have a history of logs. Or you could send off the result as email to yourself or to the team. Also, only a little magic is needed to decipher the actual result of the build from the build log and paint the mail red or green depending on the result. Once you have the basic script going, you’ll quickly find ways to improve your engineering automation.

With this simple automation we've established the basic heartbeat of our application development. Many other parts are needed to complete the picture. I’ll share some insights on those in later posts.

Comments

• Anonymous
  March 12, 2014
  Nice post. Helped provide some good insight.