Adding gulp to Visual Studio Online Builds
At one of our Code in the Cloud camps someone asked me how they could integrate front-end build tasks in the Visual Studio Online (VSO) build process. Lately we’ve been adding great support for gulp/grunt/bower into our Visual Studio IDE (check out this post from Scott Hanselman to learn more). But I have to admit that I’d never included this in a VSO Team Build. In this post I’ll show you how you can integrate a front-end build process in Visual Studio Online Build.
Disclaimer: the process described below is a demonstration of how you can integrate gulp into a Visual Studio Online Team Build process. This is by no means intended to be a best practice and to keep things simple, below scripts lack extensive error handling.
Update: in clientcompile.bat, use CD with /D option to also switch drive letter if needed (thanks Duncan!).
Step 1: Creating a Visual Studio Online Team Project
Visual Studio Online is Microsoft’s application lifecycle management solution in the cloud. It allows you to manage all aspects of you development process, from planning what needs to be done, to source control to manage your source code files, to hosted build infrastructure and even load test infrastructure. Visual Studio Online is free for teams of up to 5 members. In the context of this post, we’ll be using the source control repository and hosted build services of VSO.
Once you’re signed up for Visual Studio Online, you can create your first Team Project. In this blog post we’ll be using git-based source control; this will allow you to connect to the source code repository using any existing git client. Note that VSO source control can be used for any development technology and programming language.
To create a new Team Project, navigate to your VSO account (.visualstudio.com">.visualstudio.com">.visualstudio.com">.visualstudio.com">https://<account>.visualstudio.com) and clicking the ‘New…’ link. Enter the details and press the ‘Create Project’ button.
Step 2: Setting up the Web Application project
To keep things simple, we’ll create a basic Lorem Ipsum generator website using ASP.NET and Visual Studio, however you could use any other web technology.
Make sure that in Visual Studio you’re connected to the Team Project (from Team Explorer, Connect to Team Projects 
, and then Select Team Projects…). In Team Explorer, click the Home button and press the New… link to create a new solution that is added to source control. Now create an empty ASP.NET Web Application project.
Secondly, add an HTML file ‘index.html’ to the project. As you press the ‘Submit’ button, the generateLoremText JavaScript function is invoked. This function is included in a separate JS file, which will minimize using gulp.
1: <!DOCTYPE html>
2: <html>
3: <head>
4: <script src="js/lorem.min.js"></script>
5: </head>
6: <body>
7: <h1>Lorem Ipsum Generator</h1>
8:
9: <p>How many paragraphs do you need (1-100):</p>
10:
11: <input id="countInput" type="number">
12:
13: <button type="button" onclick="generateLoremText()">Submit</button>
14:
15: <p id="validationText"></p>
16:
17: <hr />
18:
19: <p id="loremText"></p>
20:
21: </body>
22: </html>
Now add a folder ‘js’ to the project, which will contain our client-side JavaScript code. Then add a new JavaScript file ‘lorem.js’ to this folder. The script is fairly easy and self-explanatory.
1: function generateLoremText() {
2: var count, validationText, loremText;
3:
4: loremText = "";
5: validationText = "";
6:
7: count = document.getElementById("countInput").value;
8:
9: if (isNaN(count) || count < 1 || count > 100) {
10: validationText = "Invalid input";
11: } else {
12: loremText = createLoremText(count);
13: }
14:
15: document.getElementById("validationText").innerHTML = validationText;
16: document.getElementById("loremText").innerHTML = loremText;
17: }
18:
19: function createLoremText(count)
20: {
21: var loremText = "";
22:
23: for (i = 0; i < count; i++)
24: {
25: loremText = loremText + "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.";
26: }
27:
28: return loremText;
29: }
The project structure should now look something like this:
Step 3: Configuring gulp for your web application
We’re going to use gulp to perform our front-end build tasks; you could just as well integrate with grunt or bower. As mentioned before, Visual Studio already provides grunt/gulp/bower integration in the IDE, however this will run these build tasks on the developer machine and not as part of our centralized build process.
There are basically 2 approaches for customizing the Visual Studio Team Build process:
- Create a custom Team Build template to include activities for invoking the front-end build tasks. The team at MS OpenTech has created a custom build template and corresponding activities to invoke Grunt as part of your Team Build.
- Alternatively, the default Build templates in TFS and Visual Studio Online now have the possibility to invoke pre and post build scripts. In this script you can then invoke the grunt/gulp/bower commands.
In this blog post we’ll take this second option as it allows to quickly extend the default build process without the overhead of creating a custom build template.
To achieve this, we are going to add 3 files to our Web Application project:
- gulpfile.js: contains the gulp script describing all build tasks that need to be performed.
- package.json: contains the list package dependencies for the gulp script
- clientcompile.bat (note: you can also use PowerShell): batch script that install the dependency packages and then invokes the gulp command
Your project structure should now look like this:
The gulpfile.js file contains the different gulp tasks – if you require plugins, make sure to add the necessary dependencies in the package.json file. Below script will minimize (uglify) our lorem.js script and rename it to lorem.min.js.
You’ll notice that I’m passing in a parameter (PATH) that defines the output path for the minimized script – I’m using the yargs gulp plugin for this. This path will resolve to the ‘_PublishedWebsites/…’ folder on the Team Build drop location.
1: 'use strict';
2:
3: var gulp = require('gulp'),
4: rename = require('gulp-rename'),
5: uglify = require('gulp-uglify'),
6: argv = require('yargs').argv;
7:
8: var DEST = argv.PATH;
9: if (!DEST) { DEST = '.'; }
10:
11: gulp.task('default', function() {
12: return gulp.src('**/js/lorem.js')
13: // This will minify and rename to lorem.min.js
14: .pipe(uglify())
15: .pipe(rename({ extname: '.min.js' }))
16: .pipe(gulp.dest(DEST));
17: });
18:
Our package.json file will contain the necessary packages to be installed:
1: {
2: "name": "GruntVSO",
3: "version": "1.0.0",
4: "description": "Grunt VSO integration sample",
5: "main": "index.js",
6: "author": "Nick Trogh",
7: "license": "MIT",
8: "repository": { "url": "https://github.com" },
9: "readme": "Grunt VSO integration sample",
10: "dependencies": {
11: "gulp": "^3.8.11",
12: "gulp-rename": "^1.2.0",
13: "gulp-uglify": "^1.1.0",
14: "yargs": "^3.4.5"
15: }
16: }
Finally the clientcompile.bat file will invoke the different pieces of the front-end build process:
1: REM Script arguments
2: REM [1] project source folder
3: REM [2] output folder
4:
5: REM Create the npm folder in the user profile to fix a bug with npm on Windows
6: MKDIR C:\Users\buildguest\AppData\Roaming\npm
7:
8: REM Set the working folder to the project source folder to correctly install npm
9: packages locally
10: CD /D "%TF_BUILD_SOURCESDIRECTORY%%1"
11:
12: REM Install all dependencies through npm
13: CALL npm install
14:
15: REM Invoke gulp, passing in the output folder for the minimized JS scripts
16: CALL node_modules\.bin\gulp --PATH "%TF_BUILD_BINARIESDIRECTORY%\%2"
The batch script takes 2 input parameters, which will provide in our Team Build Definition. The first parameter defines the path to the sources of our Web Application project on the build server. The second parameter defines the output folder to write our minimized JavaScript file to. Both input parameters are relative paths and are made absolute based on TFS Build environment variables.
The first step in the script (line 6) creates a folder for npm in the user profile – this is a workaround for a bug with the npm installation on Windows. On line 11, the node package manager is then invoked to install all dependencies locally (in the node_modules folder); it takes our package.json file into account for this.
After installing the packages, we can now invoke gulp and pass in the output path for the minimized files (using ––PATH).
Make sure to commit and push your changes into source control once ready.
Step 4: Setting up a team build
The final step to be done is set up our Team Build for our Web Application – this is achieved by creating a new build definition. The build definition describes the different steps that make up our build process: getting our sources from source control, compilation, invoking our front-end build tasks, and finally copying the output to the drop folder.
To create a new build definition, go to Visual Studio, navigate to the Team Explorer and open the Builds section.
This view shows the existing build definitions and also shows past build execution results. Create a new build definition by clicking the ‘New Build Definition’ link. In the Trigger tab you can choose to setup a manual build or continuous integration build.
To invoke our script for front-end build tasks (clientcompile.bat), we need to configure the build process. Open the ‘Process’ tab and expand section ‘5. Advanced’ inside the Build section. Specifically we’re going to provide the ‘Post-build script’ information. The script will be executed on the build infrastructure after the source code has been compiled, hence post-build.
For the script, navigate to the clientcompile.bat file. For the script arguments, we need to provide 2 values, as described above:
- The relative source code folder for our project: \WebApplication1\WebApplication1
- The relative output folder: _PublishedWebsites\WebApplication1 (if you choose ‘SingleFolder’ for output location, Team Build copies all output for a Web Application to the _PublishedWebsites folder on the drop location.
This is the only configuration we need to perform on our Build Definition to invoke our JavaScript build tasks. All we need to do now is queue a new build, or perform a source code modification in case of continuous integration. You can queue a build by right-clicking on the build definition.
Once our build has completed, we can check the output of the build by downloading the contents of the drop location. You do so by right-clicking the build result in Visual Studio and choosing ‘Open in Browser’. On the Build output details page you can then download the output of the build. In addition, you can view a detailed output of the build process in the ‘Diagnostics’ tab.
When you download the build output zip file, you will find the lorem.min.js file inside the js folder. Opening the file in Visual Studio or a text editor shows the minimized contents of the original lorem.js file.
Summary
As you can see, using the pre and post build scripts you can very easily integrate custom logic into a Visual Studio Online Team Build process. By adding your front-end build process into your central build/continuous integration process, you can have the same repeatable and controlled process for both server-side and client-side builds.
In a next post I’ll explain how you can include front-end build tasks into your Azure Websites continuous deployment process.
Additional Resources
- Come join us for a Code in the Cloud developer camp to learn more about Visual Studio or attend one of our other developer camps.
- Check out our free online trainings in Microsoft Virtual Academy
- Get your free Visual Studio Online account and free Visual Studio Community IDE
Comments
Anonymous
March 05, 2015
I'd usecd /d {path}in case you need to change drive letter, i.e. CD /D "%TF_BUILD_SOURCESDIRECTORY%%1"Anonymous
March 05, 2015
@Duncan: thanks for the great tip. I've updated the post accordingly.Anonymous
March 16, 2015
There is no Post Build Script Path, Post Build Script Arguments fields nor anything for PreBuild either. What template are you using?Anonymous
March 16, 2015
I take it back I'd gone to 5. Advanced off the Root menu, not 5. Advanced WITHIN 2. Build section.Anonymous
March 17, 2015
Cool approach, thanks for sharing. I recently solved the problem utilizing a third approach - extending the build targets within the .csproj file. If you're interested, check it out here: www.codecadwallader.com/.../integrating-gulp-into-your-tfs-builds-and-web-deployAnonymous
March 17, 2015
Adding support for this in the IDE is really great, but I think the Build side of this is a bit ignored. I really hope this will be a lot easier in TFS2015 (or VSO next). From the screenshots I've seen, there is definitely improvement.Anonymous
March 17, 2015
@Steve: nice article and creative solution - thanks! I wanted to provide a solution that works for code that is not necessarily inside a VS project, for example when using Eclipse. In that case you can't revert to your solution.Anonymous
March 17, 2015
@Nick: Thanks. :) You're right about my solution being VS project dependent. I've added a link to your article into my post to help point people over here if my solution won't work for them.Anonymous
April 16, 2015
Thanks for your article, Nick. I have tried to set a VSO build up, to run npm install, bower install and then gulp through a bat file, just like you do. But it stops during bower install with the error message "bower bootstrap-sass-official#~3.2.0 ENOGIT git is not installed or not in the PATH". Do you have any ideas? Do I have to have git in my package.json to make bower work?Anonymous
April 16, 2015
@Søren: the Hosted build controller that comes with VSO contains Git for Windows 1.9.5 (see www.visualstudio.com/.../hosted-build-controller-vs). However, there seems to be a problem with it not being included in the PATH env variable. You can add the path to git.exe to PATH or, as you suggested, include Git as part of the dependencies in package.json. The latter approach allows you to use your git version of choice.Anonymous
April 16, 2015
Thanks Nick. Adding Git to my dependencies got me a bit further, but still getting errors. d:asrcWebnode_modulesgulpnode_modulesliftoffindex.js:171 throw err; ^ Error: EPERM, ftruncateAnonymous
February 01, 2017
Hi, can you use 'npm run build' instead of gulp to?