FAST meets SharePoint - What's Coming in Search for SharePoint 2010
Last week was the 2009 SharePoint Conference in Las Vegas. The sold-out attendance of 7400 doubled the number from the previous SharePoint conference 1 ½ years ago. This is not too surprising given the incredible momentum of SharePoint and the fact that much of the event was dedicated to disclosure of the highly anticipated SharePoint 2010 release. Surprising or not, it was gratifying for us search guys to see the level of interest in the new search capabilities being disclosed for 2010. Several of the search-specific break-out sessions had as many people in the audience (>1000) as the entire attendance of our FASTforward’09 search conference in February earlier this year.
At that FASTforward conference in February, Microsoft announced plans to deliver enterprise search targeting two general solution areas: 1) business productivity applications, where the emphasis is on search driving employee efficiency, and 2) Internet business applications, where search is used to drive customer service and revenue. The disclosure of the new search options in SharePoint 2010 at last weeks SharePoint Conference amounts to the first deliverable of this strategy.
SharePoint as a whole has evolved from the original content management and portal platform of earlier releases into a complete “business collaboration platform”, and there are *a lot* of enhancements and new capabilities in SharePoint 2010. I won’t even attempt to summarize them all here. Instead, check out Jeff Teper’s blog post from early last week, which provides a remarkably good summary of everything that’s coming in SharePoint 2010. As Jeff points out in his blog, search is just one of several major categories of capabilities in SharePoint 2010, but “enterprise search is a big investment area for Microsoft” and an area where “we’ve added depth at all levels in 2010”.
There are two main enterprise search options coming with the SharePoint 2010 release:
1) SharePoint Server 2010 Search – the out-of-the-box SharePoint search for enterprise deployments.
2) FAST Search Server 2010 for SharePoint – a brand new add-on product based on the FAST search technology that combines the best of FAST’s high-end search capabilities with the best of SharePoint.
SharePoint Server 2010 Search represents an important upgrade to the existing search for SharePoint, while FAST Search for SharePoint 2010 is a completely new offering and the first new product based on the FAST technology since FAST was acquired by Microsoft in April 2008. Customers and partners familiar with search in previous versions of SharePoint will see many important improvements in 2010, regardless of which product they deploy. For example, there is a new People Search feature for expertise identification and search-driven collaboration, to name just one (see Jeff’s post for a good summary of these general improvements).
FAST Search for SharePoint 2010 adds a whole new level of search capabilities that are a superset of what comes in the out-of-the-box SharePoint 2010 Search option. Since there are now two search options in 2010, it’s useful to understand what is unique in FAST Search for SharePoint and when you might consider using it over the out-of-the-box SharePoint 2010 search. With that in mind, here are my 10 reasons to consider FAST Search for SharePoint 2010:
1) Content Processing Pipeline
For people familiar with the FAST Enterprise Search Platform (ESP), the good news is that the most valued capabilities of ESP have been brought into FAST Search for SharePoint 2010 and made easier to access and deploy through tight integration with the SharePoint management and development tools. The open framework in FAST ESP for creating custom content processing pipelines is a good example. Since it was first introduced in version 3 way back in 2002, FAST customers and partners have leveraged advanced content processing and advanced linguistic features to create a wide variety of novel search applications. This highly valued aspect of the FAST ESP will be available in FAST Search for SharePoint and has been architected and enhanced to take advantage of the SharePoint management interfaces and development tools like PowerShell.
2) Meta-data Extraction
Meta-data is used in search for faceted refinement, relevancy tuning, targeted queries (e.g. search only the authors field), and other general techniques designed to improve findability. The problem is that unstructured documents are often devoid of useful meta-data. The ability to automatically extract meta-data to create useful structure on otherwise unstructured documents is a feature of FAST ESP that will also available in FAST Search for SharePoint 2010. Importantly, FAST Search for SharePoint 2010 takes advantage of simple administrative tools and the concept of “managed properties” in SharePoint to support adding custom meta-data extractors very quickly.
3) Structured Data Search
Structured data search is possible with both search options in SharePoint 2010, but FAST Search for SharePoint 2010 adds an extra level of sophistication for searching data that contains numbers, dates, and other encoded and structured information. To start, the full FAST Query Language (FQL) is available to application developers who want the richness and expressiveness that FQL provides. This includes support for numeric and date data types, formula-based query operations, term weighting with the XRANK operator, and much more. Also, integration with the new Microsoft Business Data Connectivity services in 2010 means that ingesting structured data from external Line of Business applications is much easier in FAST Search for SharePoint.
4) “Deep” Refinement (Faceted Search)
Previously only available in SharePoint search through 3rd party add-ons, faceted search, called “refiners” in the default search interface (SharePoint Search Center), is now native in the out-of-box SharePoint 2010 search. FAST Search for SharePoint adds to this the ability to deliver faceted search across results sets of any size while retaining precise counts on the refinement facets. This is critical for research and analysis applications where precise counts on facets are important decision making criteria. (You can see examples of deep refiners on FAST ESP powered sites like scirus.com and dell.com.)
5) Visual Search (Document Thumbnails and Previews)
Visual document thumbnails and previewer Web Parts will be out-of-the-box with FAST Search for SharePoint 2010 to help users more quickly judge what is relevant in a search result list. This includes a graphical previewer for PowerPoint presentations based on Microsoft Silverlight that allows users to quickly find the “one slide” of interest without having to open up the entire presentation.
6) Advanced linguistics
The quality of search against text data is highly dependent on the ability to apply the right language-specific processing techniques. FAST Search for SharePoint 2010 builds on the FAST ESP heritage and Microsoft tools to include advanced language processing (linguistics) for dozens of languages, including optimized processing for Chinese/Japanese/Korean.
7) Visual best bets
SharePoint already supports the concept of search Best Bets – managed results delivered with the search for specific queries. FAST Search for SharePoint adds to this the ability to render visual best bests in the form of images and even videos. Management of search best bets, both standard and visual, is through the standard SharePoint administrative console.
8) Best-in-class development platform
FAST Search for SharePoint 2010 builds on the comprehensive development framework of SharePoint 2010. The customization options range from configuring out-of-the-box search behavior (best bets) and user interface controls (Web Parts), to extending existing functionality using public Web Part code and SharePoint Designer, to creating brand new components and functionality with the available APIs. For FAST ESP aficionados, no compromises have been made in the area of extensibility with FAST Search for SharePoint, but many of the customizations in ESP are now much easier to do.
9) Custom search experiences (per user/profile)
FAST Search for SharePoint 2010 includes the same level of relevancy tuning available to FAST ESP. It will be possible, as it is in ESP, to create custom relevancy models tuned to differences in content sources, application needs, and user contexts. User context simply means that different users can have different search “contexts” that enable experiences optimized for their specific business needs. User context can be used to set the search sources, relevance rank profile, linguistic processing features, and other search features by user or user group. In an enterprise search setting, this means that a Sales Director does not have to see the exact same results as a Product Designer for a given query, even if they are searching the same sources.
10) Extreme Scale and Performance
Scale and performance of the out-of-the-box SharePoint 2010 Search has been dramatically improved – with proven scalability to 100 million documents and more. For FAST Search for SharePoint 2010, the exact same scale-out model that exists in FAST ESP has been preserved to enable extremes of content (e.g. number of documents to search), queries (e.g. the number of queries or query rate), or both. This means search solutions that can support billions of documents and thousands of queries per second.
There is much to like about what is coming with search in SharePoint 2010 and more information than I’m able to share in one blog post. You can add to the list above the general benefits of search enhanced by all the other tools and services of the SharePoint platform - including content management, communication and collaboration, business intelligence, system administration and monitoring, application development, and so on. As I’ve pointed out in previous posts, search doesn’t exist in a vacuum, and the ability to integrate and interoperate with other business productivity functions is critical to actually acting on a search result. From this point of view, SharePoint and it’s compendium of integrated services, simply makes search better.
The first public beta for SharePoint 2010 will be available in a few weeks. This will include beta bits for the standard search in SharePoint 2010 and FAST Search for SharePoint 2010. I hope you’ll be able to try out these new search products and features. In the mean time, you can learn more about what’s coming in search for SharePoint 2010 by going to the SharePoint 2010 preview site at https://sharepoint2010.microsoft.com/.
Nate
Comments
Anonymous
November 29, 2009
Hello, Does this mean that we will no longer have a stand alone version of FAST ESP ?Anonymous
December 04, 2009
Hi Patrick, FAST ESP is still there. You can review the enterprise search offerings at http://www.microsoft.com/enterprisesearch/. Regards, NateAnonymous
January 28, 2010
What will happen to the Search Server 2008 offering? will it remain or will the users be asked to move to Sharepoint Search?Anonymous
January 30, 2010
There will be a Search Server 2010 releasing along with SharePoint 2010, which is the direct upgrade for Search Server 2008. Of course, Search Server 2010 doesn’t have all the features of SharePoint Server 2010…notably people and expertise search, which is really cool. Search Server 2010 is not on the Beta download; if you want to try things you should download Search Server 2010 express. We are incorporating the feedback from Search Server 2010 express and SharePoint 2010 search.Anonymous
March 23, 2010
The comment has been removedAnonymous
March 24, 2010
Hi Jade, FQL is available, but only through the API. The Search Center uses a simpler end-user friendly query language that is common to SharePoint Search and FAST Search for SharePoint 2010. Regards, NateAnonymous
March 29, 2010
Hi, we can download this Fast serch sever for sharepoint 2010 for free at the moment, can you tell me whether its free or need to get additional licences when it comes to the final version to deploy on a Enterprise customers. thanks, ChamilaAnonymous
March 30, 2010
Hi Chamila, An additional license will be required to use FAST Search Server for SharePoint 2010 when it is available. Please contact your Microsoft account manager for more details. You can also learn more about licensing here: http://www.microsoft.com/licensing/. Click the "Contact a Specialist" option on the right. Regards, NateAnonymous
April 12, 2010
We have been thinking that this technology would be a good one for meeting a companies records management needs. what are your thoughts on this