Right Corner: WiX Heat, Left Corner: Thousands of Files
Cross posted from Brian Rogers: Removing complexity slowly.
With the Windows Installer XML Toolset (WiX) v3.0 on the verge of hitting production more and more people are starting to take a look at it and see what new, cool tools we have. One, of many, new ones is Heat.exe. This tool is the replacement (in many ways) to tallow.exe from the previous release. The tool had a complete rewrite and is able to offer some great new functionality.
As with every new tool, once someone gets a hold of it they want it to do more. In the case I don’t disagree with them.
- Was Heat designed to help automate the creation of WiX source? Yes.
- Was the intention to have Heat gather thousands of files quickly for Setup authors? Yes.
- Is Heat ready to be placed into an automated build system and generate patch compliant MSI packages? No.
This is where I am seeing the initial pain points. Setup authors that have installations with 1,000+ files don’t want to be hand coding or manipulating the output in order to fit their needs. I, being one of those authors (200,000+ files currently), completely agree. This is why, as the current feature owner, it pains me to say that Heat is not ready for this type of primetime work straight out of the box.
There are solutions, however, that can help us large setup authors out. Before I go into details on that, take a read over a response I made to someone that asked a few of these questions.
Hey Nick,
I understand your frustration, in fact have expressed them myself. Bottom line is no, Heat currently isn't setup to handle this volume of files in a fully Windows Installer compliant way. This is a known design limitation for the v3.0 release. However, I wouldn't throw the tool away. The general issue around large installations is not just getting the files onto the machine but then servicing the installation "in-place". This would mean not doing a full uninstall and reinstall for each new release. In order to handle this functionality many things need to stay the same for Windows Installer to correctly build patches for an MSI. Component key files must say the same and pretty much cannot be deleted, Feature/Component/File Ids must say the same, Component GUIDs must stay the same according to their key, DiskId's must say the same and I might be missing one or two things from that.
> Is Heat not designed to reduce the task of including 3000 files to adding a <ComponentGroupRef> (or whatever) line to a "Feature" ?
Yes and no. What you are seeing with the 1600 components in Candle/Light is actually thrown in Heat as well as a warning. It is very possible this should be upgraded to an error but I thought that was too strong. It is really up to the developer to decide that risk. However, with over 20,000 files one would probably not even see the warning. There are some ways to work around this if you are able to break up your directly harvesting into blocks of less than 1600 files.
> Why does the Heat output have one (blank, and duplicated) DirectoryRef each time there is a component in the one directory? Wouldn't this degrade performance...
These actually aren't blank, they are correctly fragmented. You can use an "-sfrag" command line switch to get rid of this functionality. This has been done to allow components to be pulled into other ComponentGroups/Features without taking all other Directory references. This is in line with with how WiX can be authored for flexibility
> Is there a way to tell Heat that the path to the files is ..\Whatever, NOT a variable, NOT the silly string it uses by default?
The answer is no. The use of SourceDir\ was decided to remove root directory path dependencies. This is especially useful in large build environments where the *.wxs files can be copied between machines. The use of the "-b" command line switch in Light allows for root directories to be substituted in place of the SourceDir. It also allows Heat to generate the Directory/Component/File Ids in a repeatable manner to help enable patching.
The technical challenges for all of this are actually not extremely difficult. However, they aren't easy either. Much thought has been done in the WiX v3.0 timeline in order to being addressing many of these issues and make this high level of generation possible. Using Extensions to Heat there are ways to do this correctly. I am currently working on doing this for a product that is deploying over 200,000 files which we would like to perform "in-place" patching on once released. I am currently using an internal prototype to deliver this functionality and hope to be able to get this extension onto the WixContrib site within the next few months. Again, automating the initial gathering of files is not the challenge. It is the doing it again for the patch that will come out after your RTM.
What can be done to enable this type of requirement?
- Layout Your Build Output into Groups of Less Than 1600 Files
- I am aware that many people will not find this to be a viable options, but it would work.
- Heat could be run against these smaller grouped directories and easily handle your needs.
- This would also enable auditing of the files your company is releasing out the door.
- Why does this matter? In one company I worked for a developer on his way out of the company got upset and wanted to deface a public website with an addition of a new GIF (a Cannabis leaf). The file was named “pot.gif” but that isn’t enough to throw off any major alarms. However, because we had put in change control on what files were allowed to ship we found this through auditing.
- Write a Custom Heat Extension
- Heat, like the other tools in Windows Installer XML Toolset (WiX), can be extended.
- Writing your own Heat extension can be challenging for some but easy for others. If you are up to the task then this could be your quickest solution.
- If you take this approach, be sure to follow Component rules and all the other fun stuff that is required to remain stable.
- Wait Until Someone Else Does the Above Mentioned Possibility
- This could be the simplest but not the most quickly available.
- If your company isn’t into letting you write your own code, this would be your option.
- Wait Until WiX vNext
- The least optimal solution would be to wait until the next release of Heat.
- Rob and I have tossed around some ideas on how to improve Heat overall in order to make it a production ready build environment tool.
- When will this happen? It will be a while <grin/>
Some good news with all of this. For my day job, I am looking into doing much of this in order to get our products out the door and patchable. I would say the contribution to WixContribwill be several months off until something is ready, but it IS coming.
Some references: