Enterprise Search and Bing Services – Part 1: The Bing Translator

(In May of this year, Microsoft launched its Bing search service for the Web. While Bing has shown steady growth in the Web search market, it’s not well known that Bing also includes a collection of services that can be accessed programmatically to enhance enterprise applications. This is the first of series of guest posts that explore how to combine some of these Bing services with Microsoft enterprise search offerings. nt)

If your organization has customers or employees in multiple regions around the world, chances are you have the need to search across content in multiple languages. Earlier this year Microsoft Research announced the Bing Translator AJAX API (https://www.microsofttranslator.com/dev/ajax/) – an interface that enables developers to integrate the translator into any Web-based application. In this article we will take a high level look at how to integrate the Bing Translator with enterprise search – specifically with the FAST Enterprise Search Platform (ESP).

The Bing Translator AJAX API is a remote service that currently supports 20 different languages. The API features include auto detection of language as well as translation between any two languages. For applications with secure data, the API supports the HTTPS protocol for secure connections over the Internet.

The screenshot below shows one example of a FAST ESP powered search application with results containing documents from different languages.

In this example, not only is the user’s query translated and expanded to include other languages (French, German, and Chinese), but the user has the ability to translate the teasers or the entire document using the Bing Translator. The search results also include query highlighting for each of the multiple translations of the query. Finally, the user can use the slider bar (or the visual navigator) to favor documents written in certain languages. Any slider action causes the result set to update automatically. The relevance control behind this slider widget is actually a feature of FAST ESP, but it shows another way of surfacing cross-lingual search.

There are various ways to display and expose query translation features to an end user, and the example above is just one. While this example applies query translation automatically, it’s better, in our experience, to allow the user to select it as an option. Alternatively, the application can display translated results in separate tabs.

Integrating with FAST ESP

This example implementation integrating the Bing Translator was done as a Query Transformer (QT) in the FAST Query Result Server (QRServer). Depending on the query, the query transformer can also suggest query translations to the user. Implementing the translator as a query transformer means it can be used with any of the supported ESP search API’s, across multiple UX implementations, and is platform independent.

query = office hours, translate = on, language = en

The query transformer changes the above input query to the multiple variations of the terms based on its original language. The input language is known since the users are usually coming from a regional portal or have a language preference set in their browser. The QT gets back multiple terms from the Bing Translator by connecting to the API through remote services (over HTTP). The original query is then expanded to search for all translated terms. All query terms are cached to minimize traffic going over the wire. Any other FAST query time linguistics features, like stemming, spell checking, and synonyms, will still apply on the translated terms.

This is just one example of integrating enterprise search with Bing services. If you’re interested in including this particular capability in your search application, or you have any more questions, please feel to reach out to me at Runar.Olsen@Microsoft.com.

Runar Olsen, Senior Architect | Microsoft Enterprise Search Group, Services

Comments

• Anonymous
  September 23, 2009
  The Bing service has to be pretty reliable in order to put it in the QR server. For a point of service search application this seems like a bad idea. The translation should be tops quarter second imo, so with a timeout and caching it still might work ;) Hence I'm leaning more towards a manual translation of hits in the UI, where it's easier to justify a delay.

• Anonymous
  October 30, 2009
  The comment has been removed

• Anonymous
  May 07, 2010
  I also had 'problems' in trying to translate directly in the query, while in microsofttranslator the translation went fine. Compared to Google it was pretty much the same, which is quite good, considering that Slovenian language isn't easy language.

• Anonymous
  July 17, 2010
  I wonder what happens when a word translates into two or more different words in a different language.  How does the translator know which word to use in the translated query?