Colleagues, Social Distance & Relevance in People Search; Social Networking tools

This blog posting applies to Microsoft Office SharePoint Server 2007.

MOSS provides some very powerful features to enable the new buzzword "social networking."  Discussions about these can be found pretty easily, and I give an overview of some tools near the end of this posting.  But specifics about how some of the details work has been devilishly hard to find... until now.  Through discussions with various people in-the-know, I've been able to assemble answers to common questions around People Search and MySite web parts.

Social Distance

By default, the results on the People Search page are ordered by Social Distance.  What the heck is that?  It's an ordering of results based on colleague relationships.  Here's how it's computed:

• Search results are always returned to the search results page sorted by relevance.
• When viewing results by social distance, additional processing on the search results page is used to group the results:
  • The first 3 pages of search results are grouped by colleague-ness: first your colleagues appear, then colleagues of your colleagues, then everyone else.
  • Within each group, the ordering is still by relevance.
  • When paging through results, another 3 pages of results will be grouped once you reach page 4, then page 7, etc.

By default, 10 results appear per page, so this groups people in batches of 30.  If you customized the People Search results page to show 25 results per page, this would group people in batches of 75.

There are potentially some quirks with this algorithm that was designed to give the best value in a lightweight manner.  Your colleagues will appear at the top of the list on page 1, but if you have lots of colleagues, more of them might appear on page 4.  This is because the colleague grouping takes place on the client, starting with a relevance ranking.  So if some of your colleagues are relevance-ranked after 3 pages worth of results, these will be grouped once you get to the next set of 3 pages (page 4, page 7, etc).  Based on relevance, these people should be less relevant to your query.

The People Search results page allows you to toggle the results ordering to only Relevance, ignoring the Social Distance grouping.

Relevance

Relevance ranking in people search is the same as that used in regular search across lists and documents.  I'm told that the ranking algorithm MOSS uses in all cases is BM25F. 

Conceptually, a user's profile is treated as a document about that person with their name as the title.  At a basic level, the more search query terms that appear in the person's profile, the more relevant the result.  Matches against a few special fields (e.g. name, alias) are returned in the high confidence webpart.

Note that the names of discussion lists that a user belongs to are included in that user's profile information.  This means that a people search for "rock climbers" should return all of the people who are members of a distribution list named "Rock Climbers at Contoso" as well as anyone who has the phrase "rock climbers" in their profile information, such as in their "About Me" field.

Colleagues

Here's where the real value of social distance and other meaningful relationships is computed.

A list of your colleagues is stored in the profile database.  During a profile import, this list is initially populated with a set of "immediate colleagues" that are computed from your profile properties: manager, peers and direct reports.  This way, when you first look at your Colleagues web part, it's not blank. 

Then, from an editing page on your Profile, you can edit your colleagues list to add & remove specific people, and to group them and set their visibility to other people.  Also from here, you can ask SharePoint to suggest other colleagues, and these suggestions are collected from several places.

My Microsoft badge photo, 1999! From your My Site page (or your Profile page, which is available even if My Sites are turned off), click Colleagues to edit your colleague list.

Where do colleagues come from?

• Immediate colleagues
  ...automatic: during profile import; comprised of:
  • Your manager
  • Your peers (others who report to your manager)
  • Your direct reports
• Colleagues added by you
  ...manual: via editing page on your Profile
• Suggested colleagues
  ...lookup: manually guided via editing page on your Profile; comprised of:
  • Sent items in Outlook. Periodically (every 5 days or so, depending on usage) Outlook analyzes Sent Items for common recipients, weighted by frequency of contacts and other factors. This is how Outlook is able to suggest likely recipients when you begin typing their name in the To: line, and in my experience, it seems to work pretty well.  When you click Suggested colleagues, it collects these names from Outlook via an ActiveX control, and adds them to the list of suggestions.
  • Office Communicator contacts are examined; that is, those contacts that you've added to your Office Communicator client from the company directory.  These are also collected from the Communicator client via an ActiveX control.
  • Site Memberships are all sites in which you are explicitly included in the "~Members" group for the site (where ~ is your site name).  Other users who are also explicitly included in the ~Members group on those sites are suggested as your colleagues.  This information is collected by a WSS profile synchronization timer job.
  • DG (Distribution Group) Memberships are examined for suggestions.  Distribution groups stored in AD are collected during the AD import.

Colleagues from these sources are collected and the top 20 or so are displayed as suggestions when you click "Add Colleague" on your colleague editing page.  It's not fruitful to pursue a more detailed understanding of this calculation, because the recommendation for improving the "quality" of colleagues found is the same: people associated with items you access more often are more likely to be suggested as your colleagues.  If you don't like the suggestions, you can remove individuals from the suggestion list one by one or all at once from this page, then get a new list of suggestions.

Privacy

Whoa!  Is SharePoint looking at my email?  I don't want that, how do I prevent it?

First, SharePoint's not looking at your emails.  It gets information about who you sent emails to by asking Outlook for that information.  Second, it will never do this without asking your permission.  You can always say no:

If you say Yes, then SharePoint will assemble a list of suggested colleagues for you, and you can accept or reject these suggestions.  But even if you say No, you can still edit your colleagues to search for and add new people, and to set the privacy of who should see that they are your colleague (Everyone, My Colleagues, My Workgroup, My Manager and Only Me):

Social Networking tools

Why did we go through all this trouble to try and figure out who your colleagues are?  Social Networking, baby!  We spent a good bit of time on the inputs and the outputs of this to make it a really useful function, instead of just paying lip service to it.

The inputs are the information about who your colleagues really are.  A simple option would have been to just provide a place for you to register your colleagues, and not go through all this supposedly intelligent guesswork.  But that would have made it almost useless, because most people are just not going to take the time to populate this list, and even fewer will maintain it as their colleagues change over time, which is natural. 

SharePoint uses the information already embedded in your company's infrastructure and the changing data you deal with on a daily basis to help you connect with the right people.

Sounds like market-speak, but that's really the goal and I think it does an amazing job.

The outputs are the tools that SharePoint can provide based on this colleague information.  Consider some of these details:

• People Search ranking by Social Distance.  By default, the top people returned when you search for a name will be the people you most likely want to find: people you've communicated with recently, perhaps as a participant on an email thread or a collaborator on a site, and now you want to find more information about them.
   

• In Common with You web part on My Profile page.  When you view someone else's profile (for example, by clicking their People Search result), this web part shows you a list of people that you both may have in common.  This is useful to quickly understand business relationships that you may need to know about.  Categories shown are

  • Manager we both report under
  • Colleagues we both know
  • Memberships we both share

   

• Colleague Tracker web part on MySite.  This shows you recent changes to user properties of your colleagues, effectively a newsletter of what's new with people you work with.  Information you can track includes

  • Anniversaries
  • Profile property changes
  • Membership changes
  • New documents this person has posted
  • Out of Office status
  • Blog postings from this person

   

• Colleagues web part on My Profile page.  This displays your current list of colleagues, grouped into categories that you may have applied.
   

• Memberships web part on MySite.  This shows Distribution Lists and SharePoint Sites of which you are an explicit member.  For SharePoint Sites, it is rather specific: you must appear as an individual member of the ~Members group on the site (where ~ is the site name).  If you have access to the site through a different configuration, such as being a member of the ~Owners group, or being a member of an Active Directory Security Group that is a member of the ~Members group, this is not counted.  Otherwise, everyone would very likely see all the portal sites displayed as their memberships, and that would not be useful.
   

• Organization Hierarchy web part on My Profile page.  This shows your manager, your peers (others who report to your manager) and your direct reports, if you have any.

These are some of the tools that MOSS provides for Social Networking; I've highlighted the ones that use profile data and the powerful "colleague" pattern to show meaningful relationships among people.  For a better introduction to the full set, see the product team's Enabling and Managing Social Networks for Business use with MOSS.

Technorati Tags: SharePoint,Social Networking,MySites,People Search,Social Distance

Comments

• Anonymous
  May 01, 2008
  Mark, This is a great post! I just had a customer ask me why the CEO's name doesn't appear at the top of the results when searching by his name. Now I know, he wasn't a colleague of the user. I feel people are hungry for this type of "insider" information, so they can understand how MOSS works and why it does what it does. I hope you and others with inside knowledge publish more of this type of information.

• Anonymous
  May 02, 2008
  Jimmie, Thanks for the feedback.  To clarify your comment here about not finding the CEO, from our email conversation.  You discovered the CEO wasn't found because the searcher entered his last name only, which matched too many other results in the company, since the company's name was the same as his last name.  Once the searcher typed in the CEO's full name, he WAS found as the top result.

• Anonymous
  May 02, 2008
  [Cross-posted from Mark Arend's blog . Mark has been doing some exciting work with SharePoint User Profile

• Anonymous
  May 03, 2008
  Hi Mark, thanks for a great post. This goes a long to help exlain the logic behind some of the operation. In your last point you mention the Org Hierarchy. Do you know if there are any plans to fix the broken links for the assistant field on the profile page? Myself and others have previously blogged about this problem: http://blog.gavin-adams.com/2007/10/16/assistant-mapped-user-profile-property-is-broken/ Thanks, Gavin

• Anonymous
  May 05, 2008
  ToppTipp Microsoft SharePoint Administration Toolkit 1.0 ( WhitePaper ) x64: http://www.microsoft.com/downloads/details.aspx?FamilyId=F8EEA8F0-FA30-4C10-ABC9-217EEACEC9CE&displaylang=en

• Anonymous
  May 05, 2008
  ToppTipp Microsoft SharePoint Administration Toolkit 1.0 ( WhitePaper ) x64: http://www.microsoft.com

• Anonymous
  May 06, 2008
  Gavin, sorry, I don’t know the status of this one way or the other.  I did a quick search of our bug database, but didn’t find anything specifically related to this issue.  You could search list of fixes for SP1 in KB article 942390.  If I come across some information, I’ll let you know.  

• Anonymous
  May 18, 2008
  Thanks for a great post, on a topic that I can see growing a lot in the coming year.