Partager via

Hadoop for .NET Developers: Setting Up a Desktop Development Environment

  • Article

NOTE This post is one in a series on Hadoop for .NET Developers.

If you are a .NET developer, you will want to setup a desktop development environment with the following components:

  1. Visual Studio 2010 or 2012
  2. NuGet Package Installer for Visual Studio
  3. A Local, Single Node Hadoop “Cluster”

Having these components installed on your desktop will allow you to develop against Hadoop locally as well as against a remote cluster (whether on-premise on in the cloud). You might be able to get away with not installing Hadoop locally, but most of the .NET-oriented documentation I’ve found assumes this is your setup.

I will assume you are comfortable installing Visual Studio on your own and the NuGet site provides simple enough installation options. I do recommend installing Visual Studio and NuGet first and then making sure your system is up-to-date with patches before proceeding with the Hadoop installation.

The Hadoop installation is very straightforward. That said, I once experienced an installation failure that prevented subsequent re-installation attempts. To fix this, I navigated to the <System>:Users<user name>AppDataLocalMicrosoftWeb Platform InstallerInstallers folder and deleted the installer associated with the Hadoop installation. After that, I was able to successfully re-install Hadoop.

Finally, I have found installing these components to a virtual machine running on my Windows 8 desktop (using Hyper-V) to provide me a reliable environment from within which to work and keeps my base desktop environment clean and stable. Whether you install to a VM or to your base desktop, please note that if your IP address changes, you will need to restart Hadoop as some components tend to hold onto the IP address with which the cluster was started. While I am told you could selectively restart components to clear the old IP address, I do not know which ones and have found that simply rebooting my development environment does the trick quickly enough :-)

Comments

  • Anonymous
    August 20, 2013
    What about VS 2012 ?

  • Anonymous
    August 20, 2013
    Sorry for the oversight. I meant to say 2012 in the post. Its updated now.

  • Anonymous
    August 21, 2013
    It is not <System>:UsersuserLocalMicrosoftWeb Platform InstallerInstallers but <System>:UsersuserAppDataLocalMicrosoftWeb Platform InstallerInstallers (missing the AppData Fragment)

  • Anonymous
    August 22, 2013
    Thanks for catching that.  I've updated the post

  • Anonymous
    September 16, 2013
    private-repo-1.hortonworks.com/.../hdp-1.0.1-ms9.winpkg.msi is returning a 404 when the Web Platform Installer tries to download it.

  • Anonymous
    September 16, 2013
    @Sean - Sorry I don't have an answer for this one.  This would indicate there's a communication issue between the system to which you are installing the solution and the server you are downloading it from.  You might contact Hortonworks on this one: hortonworks.com/.../contact-us

  • Anonymous
    February 07, 2014
    Is there a way to set HDInsight emulator that it will bind to full machine name, not to localhost? Thanks Voffka