Share via

Building projects in parallel

  • Article

Greetings MSBuilders!

Orcas MSBuild introduces a new feature allowing build authors to build projects in parallel. To enable this feature we have introduced a new parameter to the MSBuild task called “BuildInParallel” and a new command line parameter called “maxcpucount” or “m”. Note that if you do not set the “maxcpucount”, even though you set “BuildInParallel” parameter to true, your projects will be built serially.

Each MSBuild process (including the main or parent process started from command line) contains a component called the node. Projects are processed and built by the node component. Thus using “maxcpucount” of 2 will result in 2 MSBuild processes. If your projects have dependencies on other projects that are being built in parallel, then you will need to set appropriate project to project references so that dependent projects get built first. For instance in the below example (Example 1) if project d has a dependency on project c, building them in parallel could result in any of these projects being built first or both building concurrently. If project d got built first then your build would fail. In order to prevent this you will need to set up a project to project reference from project d to project c. Thus, if project d is built first, it will first follow the project to project reference and build project c, then it would build project d. Setting a project to project reference include invoking the MSBuild task to build the dependent project. Example 2 briefly demonstrates this.

The order at which the projects are built is dependent on the scheduling algorithm of MSBuild. You cannot assume any particular order.

Example 1:

a.proj

<Project xmlns="https://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion=”3.5”>

<Target Name="default">

<MSBuild Projects="b.proj;c.proj;d.proj" BuildInParallel="true"/>

</Target>

</Project>

b.proj

<Project xmlns="https://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion=”3.5”>

<Target Name="default">

<Message Text="Building project b"/>

</Target>

</Project>

c.proj

<Project xmlns="https://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion=”3.5”>

<PropertyGroup>

<OutputType>DLL</OutputType>

<AssemblyName>c</AssemblyName>

</PropertyGroup>

<Target Name="default">

<Message Text="Building project c"/>

</Target>

</Project>

d.proj

<Project xmlns="https://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion=”3.5”>

<PropertyGroup>

<OutputType>EXE</OutputType>

<AssemblyName>d</AssemblyName>

</PropertyGroup>

<ItemGroup>

<Reference Include=“c”/>

</ItemGroup>

<Target Name="default">

<Message Text="Building project d"/>

</Target>

</Project>

Command line: msbuild a.proj /m:2

In order to prevent the build from failing in case project c and project d build simultaneously we need to set up the following project to project reference

Example 2

c.proj

<Project xmlns="https://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion=”3.5”>

<PropertyGroup>

<OutputType>DLL</OutputType>

<AssemblyName>c</AssemblyName>

</PropertyGroup>

<Target Name="default">

<Message Text="Building project c"/>

</Target>

</Project>

d.proj

<Project xmlns="https://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion=”3.5”>

<PropertyGroup>

<OutputType>EXE</OutputType>

<AssemblyName>d</AssemblyName>

</PropertyGroup>

<ItemGroup>

<ProjectReference Include=”c.proj”/>

</ItemGroup>

<Target Name="default" DependsOnTargets=”BuildProjectReferences”>

<Message Text="Building project d"/>

</Target>

<Target Name=" BuildProjectReferences ">

<MSBuild Projects=”@(ProjectReference)” />

</Target>

</Project>

[Author: Jay, MSBuild Software Engineer]

Comments

  • Anonymous
    June 04, 2007
    I will build solutions (their projects) in paralell too? Thanks, Rui

  • Anonymous
    June 15, 2007
    Is this parameter available already? If not then when it will be available?

  • Anonymous
    June 15, 2007
    DDG, you need to be using the msbuild executable shipped in the 3.5 Framework (currently beta 1). I set it up on the project I'm currently working on -- it knocked about 3 seconds off of a full build (12 seconds vs. 9 seconds on a low end T2060-based laptop). Not bad -- as we add more modules, I expect this could notch up a bit more.

  • Anonymous
    June 20, 2007
    MSBuild is introducing a new feature in Orcas where projects can be built in parallel - see their team

  • Anonymous
    June 26, 2007
    The comment has been removed

  • Anonymous
    June 26, 2007
    DDG, This parameter will be available in the next version of .Net framework (Version 3.5). You can also get this version by installing Visual Studio (code name Orcas) beta versions. Thanks, Jay Shrestha (jaysh@microsoft.com)

  • Anonymous
    July 17, 2007
    Greetings MSBuilders! Passing properties to projects specifically when you want to pass different sets

  • Anonymous
    July 17, 2007
    Greetings MSBuilders! In Visual Studio 2005 you could only target the 2.0 Framework tools. MSBuild targets

  • Anonymous
    July 20, 2007
    While some of the build agent properties are available in the VS GUI, buried in the tfsbuildservice.exe.config

  • Anonymous
    July 20, 2007
    While some of the build agent properties are available in the VS GUI, buried in the tfsbuildservice.exe

  • Anonymous
    November 11, 2007
    Earlier this week, I spent some quality time porting a moderately complicated build over to MSBUILD 3.5,

  • Anonymous
    April 14, 2008
    Is there any reason why Microsoft is not able to do the compilation of source to obj files in parallel, like GNU make does since years? I'd consider this the first and easiest step to improve build performance, especially with the rise of multi-core processors.

  • Anonymous
    May 15, 2008
    You mean like with <a href="http://msdn.microsoft.com/en-us/library/bb385193.aspx">the /MP compile option</a>? It even worked in 2005, although it was undocumented and unsupported, and quite possibly buggy.

  • Anonymous
    June 10, 2008
    Try to use codeblocks. It supports parallel build.

  • Anonymous
    June 26, 2008
    With Visual Studio Team Build 2008 there is no way to run two builds from within the same Team Project

  • Anonymous
    July 09, 2008
    TeamBuild can build one build definition from a single Team Project at a time. can run multiple build

  • Anonymous
    August 27, 2008
    I'm thinking of >1 MLOC C++ solutions that take hours to build. Many companies end up buying from http://www.xoreax.com I wonder how difficult it would be to plu-in a more generic grid engine like http://www.digipede.net/

  • Anonymous
    February 25, 2009
    Can you explain why provided sample is much faster without /m switch? (the same happens for a solution I'm trying to build)

  • Anonymous
    March 19, 2011
    Because of the (possibly) bug connect.microsoft.com/.../msbuild-exe-appears-to-hang-when-maxcpucount-is-used-and-a-large-number-of-project-references-exist Unfortunately, building of my large solution is much faster with single-process msbuild

  • Anonymous
    May 07, 2012
    I am facing a issue related to parallel build. As I have almost 900 projects, I am building them parallel but getting exception of file access. eg. "Unable to copy x.dll into ... as it being used by another process."