Who Can You Trust?
Awhile ago, my manager and I were kicking around ideas to solve a problem in the MSDN Forums. By default, every new thread started on the forums is tagged as a "question". The question is considered "unanswered" until a reply is posted to that question that is marked as an "answer". The answer can get marked in two ways:
- The original question asker can mark the reply as an answer.
- Any moderator on the forums can mark replies as answers.
We track our metrics on how "healthy" the forums are based on percentages of questions that have been marked as answered. The problem Josh and I were trying to find a solution for was that the extra answer-marking step just wasn't happening all of the time. Often, the original question-asker was new to the forums, and didn't understand the "Mark as Answered" button. We have very few moderators relative to the vast size of our forums community, and they simply can't read all of the questions. But somebody must be reading all of those replies...
We came up with the idea that we should give many, many more people "answer marking" rights in the forums. Why not? After a user had answered just a few questions--5 or 10, they would have the ability to mark any reply as an answer on the forums. We did a quick check, and it looked like that would make about 3% of our users "answer markers". I said: "Why hold back at all? Why can't we trust even a new forum user? Let's just give everyone answer marking rights." I went and made a post in the forums and in the DDCPX Team Blog asking customers what they thought about this idea.
What was interesting was that people generally disliked the idea. People said that replies were already marked too aggressively now--that the basic new user on the forums couldn't be trusted to know what really was an answer. Somebody even replied that the original question asker couldn't even know if their question was answered enough to mark it!
The entire thread was interesting to me--who and what do you trust in an online community? I don't mean "trust" as in letting them watch your dog while you are on vacation, but who can you trust to not spoil the the community experience? Should people be given the benefit of the doubt, until they mess up (the Wikipedia philosophy, where anyone can edit or create an article), or should they have to earn their stripes before they get those rights (the MSDN Forums current philosophy)? I firmly believe the more controlling we are with those rights, the less interested newbie users are in joining the community...but other's believe that control is a necessary evil in making sure there's true structure to the community.
As for the forums, we added a work item to our request list to the forums team to please give answer marking rights to everyone with over 5 answers contributed. Time will tell if we're going to create bedlam or not... :)
Comments
- Anonymous
February 06, 2006
The comment has been removed - Anonymous
February 07, 2006
There definitely can be a problem with people trying to "game" the system by providing curt, incomplete answers, and marking them as answered. To help prevent this, the forums allow the original question asker, or any moderator, to go ahead and "unmark" a question. No, not everybody uses this, so we definitely have a "quality of answer" problem, but I don't think it's any worse in our forums than any other technical support community.
Outside of moderation, what would you suggest as a way to ensure the quality of an answer? Right now, our approach is definitely black and white, but we're always open to creative suggestions. We have the "post is helpful" button--but the people who are voted the most helpful are typically the ones that answer the most questions. Maybe some sort of ratio between number of answers and number of helpful votes would help?
Thanks for your input...and if you'd like to become a moderator in the MSDN Forums, please send me an email through this blog with your forums display name, and I'll work on getting you access.
Joe - Anonymous
February 09, 2006
In my last post, I talked about how difficult it was to try and decide what groups of people you can... - Anonymous
February 10, 2006
Joe,
If people think less about the forum as a quick answer to a problem and more as an archive / resource for finding quality answers, the items you have in place are fine and this is a moot point.
You may want to make it more formal to ask and answer a question. Perhaps have additional fields requiring more information when asking a question and the ability to only have 2 open ( unanswered ) questions running at a time.
Answers may have a specific place to put code, links to external resources, links to similar questions in the forum that already have an answer, etc. In this way, you have a little more information to assure questions are answered well and measure good answers. By having information broken up that way you can also present and use the information ( code and links ) in a much more useful fashion.
Regards,
Dave
MVP C#
http://davidhayden.com/blog/dave/
http://codebetter.com/blogs/david.hayden/ - Anonymous
February 10, 2006
I meant to say each user is only allowed to have 2 open ( unanswered ) questions running at a time per forum. Now that I think about it that is probably unrealistic and unfriendly.
I was trying to think of a way to get people to mark questions as answered.
Dave