UK Data in the Cloud

There is still a lot of inertia in the UK about storing data in the cloud for entirely valid reasons and rather vague uncertainties and doubt.  For a few organisations keeping data in the cloud is exactly the right thing to do because those organisations want to actively share their information, the most obvious is the UK government with their data.gov.uk initiative.  Commercial companies may also want to sell their data and rather than opening up fat pipes into their data centres the logical approach is to have this hosted on a public cloud as well. 

I mention this because one aspect of Azure that Microsoft rarely talks about is the Windows Azure Marketplace (WAM), a portal where for the sharing and consumption of large data sets.  Originally this was just US based like a lot of Microsoft services, but over the last year or two it has grown steadily so that there are now a significant number of UK relevant data sets on there, most notably is the Met Office Open Weather Data (and actually part of data.gov.uk)

Some of this data you will be paying for based on how many times you query it and so one way to minimise that cost would be to download it then create my own  internal data market  which would also include sets of data from in house systems for users to mash up using tools like PowerPivot.

You can of course connect PowerPivot etc. to the WAM, and the good thing about this approach rather than just pulling down a .csv file is the connection location is remembered, and this is useful for several reasons:

• you know where the data came from so it’s verifiable
• you can easily refresh your analytical model with the attest data form a given source
• if you wish to scale up your model either to SharePoint in house SharePoint, Online in Office 365 or to a BI Semantic Model in SQL Server 2012, the source is preserved so the model can still be refreshed.

So the Azure Data Market works well with self service BI to allow analysts to develop models based on external and internal data, say for mapping the weather to sales to develop models to predict demand as I have posted before.

The other way this data can be consumed is to use it inside an application.  I can see a case for this sort of thing on a property search site where additional local information is bought in alongside the details of the house/flat you are looking for such as schools and their stats, hospital metrics, rail commute times,  and so on.  This will typically incur a cost but would give this site an edge over its competitors and possibly be recouped through advertising.  There are also applications you can integrate with such as translation services and Bing.

It’s also worth bearing in mind that you could be making money out of your data, by selling it via WAM as well.  Obviously this would not be personal data, so things like market research house price information, trends in the UK job market from a recruitment agency which have been anonymised. 

Finally there’s extensive help on how to use all aspects of WAM, such as code snippets, samples and hot to videos, and it’s changing all the time so even if there’s nothing of interest right now there may well be next time you look.