Data Feeds

One of the keys to self-service BI is that people need the highest quality, up-to-date data for their analysis. Many people don’t have direct access to query the databases underlying the applications that they use on a day to day basis, both because of the complexity of the data and also the need for adding business logic and security on top of the raw data.

Gemini includes support for data feeds, which lets you get your data from all kinds of interesting places such as applications and data services. One of the most common places where people get their data today is Reporting Services reports. So let’s start with some great news for Gemini users.

Using a Reporting Services report as a data feed

In SQL Server 2008 R2, every single Reporting Services report can be used as a data feed for your Gemini models! This means that users can get refreshable data feeds sourced from all the places where SSRS can get data, without any further work required to make data available for Gemini.

Once you have found a useful report, you can add it to your Gemini model by clicking the new orange “Export to Data Feed” button on the toolbar. This will launch Excel and take you straight into the import wizard. If you already have Excel workbooks open, you will be prompted to either pick an open model to add the feed to, or create a brand new model.

The import wizard for data feeds is very similar to the wizard for other types of data. In particular, you can preview data and pick which columns from the feed to add to your model. If the data feed contains multiple tables, such as a Reporting Services report with different data regions, then you can pick which tables to add and specify table names.

You can also start from the Gemini client, browse your report server for a useful report and add it to your Gemini model. Whether you start from the report, or start from the Gemini client, you can add as many feeds as you need to build your model, including combining it with data from other sources (see Edward’s blog on Importing data into Gemini).

Using a data feed from the Internet

Reporting Services is just one of the many places where you can get data feeds. You can also use data feeds from the Internet or other sources, as long as they follow the REST and Atom model (see next section for technical details on developing a custom data feed). One really cool place for getting data is the Open Government Data Initiative (OGDI) at https://ogdisdk.cloudapp.net/ which is built on the Azure Services Platform.

If you know the URL of the data feed, you can use the From Data Feeds button in the Gemini client to import it. You can also set up advanced properties to have more control over how the data shows up, including choosing options like whether to include all the standard Atom columns like title and author name which are not shown by default.

Developing a custom data feed

If you are a developer, you are probably already wondering how you can enable Gemini users to get data feeds from your applications. The good news is that there are lots of easy ways to support this. The basic technology is REST and Atom, so if you already have web services support you are in good shape to get started. One of the simplest ways to build a data feed is to use the great support for syndication in WCF. You can add your custom columns to the feed using the extensibility mechanisms that they supply, see one of the OGDI feeds mentioned above for an example, such as https://ogdi.cloudapp.net/v1/gsa/ConusPerDiemRates2009/ (you may need to turn off feed reader view in your browser to be able to see the raw XML).

If you are developing on the .NET platform and want to build a flexible and powerful data service, then ADO.NET Data Services (“Astoria”) is your best bet. If you haven’t used this technology yet, you will be surprised at how few lines of code it takes to get a huge amount of functionality, including everything you need for data feeds in Gemini.

Other than the data feeds themselves, if you want to add the orange data feed button to allow users to launch Gemini from your application or save a data feed file for later use, then you will also have to build a service document (.atomsvc file) for your data feeds. ADO.NET Data Services includes a service document already, but if you have built your own data feed then you can use WCF’s ServiceDocument class, or otherwise the format is simple enough that you can build one yourself.

If you developed have a desktop application rather than a server application that could include REST web services, but you still want to get data into Gemini, then you have a similar option. Gemini can import data from regular text files and the clipboard so in many cases users will already be able to use your data. But, if you want to let users launch Gemini right from your application to import data, you can create local Atom XML files and a corresponding data service document and then use the Gemini launcher to provide the same kind of seamless experience you saw in Reporting Services earlier.

Sharing and finding data feeds

If you have developed a custom data feed for your company but don’t have an existing application that users would expect to use for getting data, Gemini’s SharePoint integration includes a great solution. You can add a Data Feed Library to your SharePoint site, which will allow you to publish and edit data service documents as .atomsvc files. This means that a user could do search SharePoint for a term like “Customer data”, find your data service document in a Data Feed Library, and launch into Gemini just by clicking the file to import data from your feed.

Comments

• Anonymous
  January 24, 2010
  The comment has been removed
• Anonymous
  April 09, 2010
  PowerPivot works great with OData feeds, check out www.odata.org for more information.
• Anonymous
  May 13, 2010
  Hi, I'm trying to access a data feed that requires basic authentication. I fill in the required details on the advanced screen, but when I look at the request in fiddler no header information seems to be passed in. Am I missing something?Thanks,R.
• Anonymous
  November 05, 2010
  The comment has been removed
• Anonymous
  January 27, 2011
  The comment has been removed
• Anonymous
  February 22, 2011
  The comment has been removed
• Anonymous
  February 28, 2011
  In my case, the problem turned out to be that the feed included a reference to a report column for the report version; although I can't explain why, the fact that the report version completely changed every time the report ran, was causing the feed to break. I edited the feeds to remove the refernece to this column and they worked fine from there.