Finding the Good Stuff on the Forums

I recently had the opportunity to attend the Online Community Camp run by Forum One Communications in San Francisco this past week, a gathering of people throughout the industry that were involved in some way connecting people using the power the Internet. Some of the conference attendees were “big” companies like Microsoft, Apple, Cisco, SAP, who were thinking about community in the same way that I am: basically using the collaborative power of an online community to help people get fast and accurate answers to their questions. Other people’s organizations were based primarily around online community interactions—it was absolutely fascinating to listen to the stories that some of these people had to share about the way they were connecting people around the world with similar interests.

There were three breakout sessions that I was able to attend, and thankfully, the first breakout session of the day was about reputation in online communities. Right up my alley, right? There were a few key points that people shared, such as to remember that reputation systems are not one-size-fits-all (some communities do not lend themselves well to competition between users), and to avoid reputation systems based purely on volume (something I touched on a bit ago.)

The most interesting point brought up was that reputation systems actually have two main focuses: they can judge a user’s reputation (which is what I’ve been focusing on) or they can judge the reputation of content. Wow...completely obvious, but it’s something that I’ve nearly entirely looked over, with the exception of one lowly post were I talked about post quality.) So, how can we judge the reputation of posts, namely answers, on the MSDN Forums?

One core concept about rating post content is that people simply don’t “vote” for content without an incentive. We currently have “Helpful/Not Helpful” buttons on every post on the forums—and they are clicked at about one-third of the rate of the “Mark as Answer” buttons. Why? There’s no reason to mark something as helpful. Nothing appears to happen. Yes, we take those votes and use them to help with search on the backend, but an average user has no reason to click. So, any post rating system actually asking a user to separately “rate” the quality of a post is likely to not get a ton of involvement.

The idea that Josh proposed today during a thread with another team at Microsoft thinking about the same problem was to build the post rating directly into the answer marking. Instead of a “Mark as Answer” button, there could be a drop-down for Mark as Answer with the following options:

1. Not an Answer
2. Maybe an Answer
3. Answer
4. Good Answer
5. Excellent Answer

The polled and averaged results of all of the votes from every registered user on the site could be compiled for an overall “answer score” for the post, that would be exposed on the post itself. Every thread containing a reply post with an average above 3 would be considered an “answered” question. Each answer score would roll up directly to a user’s reputation—the higher rated their answers, the higher rated the contributor.

This answer score could be combined with the number of raw page views on an individual thread to help determine what posts are the most popular and good. A post with a high answer rating and a high number of page views is likely going to be something that people are going to want to read…what do you think?

Comments

• Anonymous
  June 01, 2006
  Excellent idea.  It's one think to provide a community and the ability to "trust" the posts of certain users; but, it's another to be able mine useful data from the community.
  
  Adding granularity to the "answers" would be a great step in being able to provide value-added to the content of the community.  I would suggest that only the original creator of the thread be able to rate the answer(s) beyond a certain level.
  
  I would further suggest that a more granular answer rating should replace the "is useful" flag, which should then be deduced from the level of the answer(s) rating(s).  After all, something deemed as being answered, IMO, is useful.
• Anonymous
  June 01, 2006
  The comment has been removed