通过

Apache Hadoop on Windows Azure Part 9 – Using Interactive JavaScript for Data Visualization

  • 项目

Apache Hadoop on Windows Azure is integrated with a web-based interactive JavaScript console, which allows you to perform

  • Perform HDFS operations, including uploading/reading files to/from the HDFS
  • Run MapReduce programs from .js scripts or JAR files, and monitor their progress
  • Run a Pig job specified using a fluent query syntax in JavaScript, and monitor its progress
  • Visualize data with graphs built using HTML5

 

Once you have built your Hadoop cluster on Windows Azure, you can launch Interactive JavaScript web console directly as below:

 

 

After the Interactive JavaScript is launched you can enter a set of commands or your own JavaScript directly from the browser:

 

 

For our example we will do the following:

  1. Upload a data file to HDFS
  2. Run JavaScript to parse the data
  3. Run Graph Visualizer with parsed data

 

Let’s collect some real time data about mobile market share for 2011:

https://www.netmarketshare.com/operating-system-market-share.aspx?qprid=9&qpcustomb=1

 

Create a txt file locally as mobilesharedata.txt locally:

 iOS 52.10 
JavaME 21.27
Android 16.29
Symbian 5.76
BlackBerry 3.51
Other 1.07

Upload this file to HDFS from Interactive JavaScript console:

js> fs.put()

File uploaded.

Verify the file is uploaded:

js> #ls

Found 2 items

drwxr-xr-x - avkash supergroup 0 2012-01-02 20:37 /user/avkash/.oink

-rw-r--r-- 3 avkash supergroup 81 2012-01-02 20:52 /user/avkash/mobilesharedata.txt

 

Let’s read the data this

js> file = fs.read("/user/avkash/mobilesharedata.txt")

iOS 52.10

JavaME 21.27

Android 16.29

Symbian 5.76

BlackBerry 3.51

Other 1.07

  

Now parse the data as below:

js> data = parse(file.data, "OS, MarketShare:long")

[

0: {

OS: "iOS"

MarketShare: 52

}

1: {

OS: "JavaME"

MarketShare: 21

}

2: {

OS: "Android"

MarketShare: 16

}

3: {

OS: "Symbian"

MarketShare: 5

}

4: {

OS: "BlackBerry"

MarketShare: 3

}

5: {

OS: "Other"

MarketShare: 1

}

]

  

Set the graph visualization settings as below:

 

js> options = { title: "Mobile OS Market Share in 2011", orientation: 20, x: "OS", y: "MarketShare" }

{

title: "Mobile OS Market Share in 2011"

orientation: 20

x: "OS"

y: "MarketShare"

}

Now let’s visualize the data in Bar Graph:

 

 js> graph.bar(data, options)

 

Now let’s visualize the data in Pie Graph:

 

js> graph.pie(data, options)

 

 

 

Keywords: Windows Azure, Hadoop, Apache, BigData, Cloud, MapReduce

Comments

  • Anonymous
    June 05, 2012
    Most of it works fine for me with the exception of the parse command. It does not give me a [ 0: { OS: "iOS" MarketShare: 52 } 1: { OS: "JavaME" MarketShare: 21 } ] type of value for me but my parse result is like this: [ 0: { OS: "iOS" MarketShare: 52 } 1: { OS: "JavaME" MarketShare: 21 }    map: [object Function]    filter: [object Function]    forEach: [object Function]    reduce: [object Function] ] As a result, the graph does not work. What should I do to avoid these        map: [object Function]    filter: [object Function]    forEach: [object Function]    reduce: [object Function] in the search results?

  • Anonymous
    August 28, 2014
    I had a question regarding the Interactive Console. After creating the storage and the HDInsight service for Hadoop, when I click on Manage Cluster for configuration changes, Hive editor opens up instead of Cluster home page which displays Interactive Console tiles,Remote Desktop tiles etc. I am new to Azure and have no idea why this is happening. Any help will be appreciated