Dealing with N00bs on the MSDN Forums
One of the most fun parts of my work with the MSDN Forums has been working some of the most passionate, smart customers we have in Developer Division—the forum moderators. The moderators are usually thinking about a month or two ahead of Josh and me, so whatever new issues they are bringing to our attention today usually come to the forefront in due time. In a way, I guess I think of our moderators as “Community Barometers”…they let us know whether or not a storm is brewing on the horizon.
That’s why I knew that there is a big element missing from my reputation system thoughts when one of our most passionate moderators, ReneeC, sent me an email earlier this week. She noticed that I’m constantly on a hunt for new community moderators, and correctly assumed that it was because I see a rising tide of questions on the site (3,100/week at last check) and knew that I needed to enable more passionate community members on the site to continue to keep pace. What she also told me was that simply adding warm bodies to the mix wasn’t going to help exacerbate one of the biggest problems with the rise of questions on the site: new users don’t know how to ask good questions.
For example, let’s say a new user to the site is really having trouble with populating a WinForm DataGrid with the results of a query on a database. A more experienced user will know to include code samples, a detailed description of what he/she is trying to accomplish, and the specific error message that they are receiving. Unfortunately, the new user on the site isn’t used to writing questions, and they often will post a message like:
Subject: Please Help!
Body: I am trying to get a DataGrid to show the results of a query I wrote and it isn’t working. What should I do? Thanks!
Ugh. That question has less than a 20% chance of being answered. In fact, we’ve found that questions written by MSDN Subscribers (typically quite experienced users) have a 75% overall answer rate, versus a 59% answer rate on the overall site! Does good question phrasing matter? You bet!
So, how do we help new users write good questions? I’m not sure, but here’s some ideas below…
1.
Welcome Messaging—We know how many posts a particular user has made on the site, and we can also figure out fairly easily what percentage of questions that they have asked have been answered. Using this information, we could target welcome messaging at new users. It could be a pop-up that displays on the homepage that links to a best practice guide to using the forums (idea c/o Renee), an email message that is sent to your Passport account after you register to the forums, or a page you must read when you go to create your first post. The content of the message will be an introduction to the forums, how they work, and guidelines to asking and answering questions. The downside to this method is that most users simply click through these guides—there’s really no incentive to actually reading them. When was the last time you read a welcome guide?
2. Question Moderation—When you first come to the site, before you’ve built up a basic reputation as a good question asker, you might have to have your questions approved by a moderator before they appear on the live site. The moderator could always tell you that you need more detailed information, and this could be coupled with the welcome messaging. The problem with this approach is twofold: 1) This requires quite a bit of extra moderation and 2) This potentially creates a feeling of distrust and extra unnecessary “hoop jumping” for some of our new users.
3. Forced Search Prior to Posting—Many of the questions that new users ask could be solved through a simple search of the forums site—saving the question asker and answers time both. If new users to the system had to search and look through a page of search results prior to seeing the post dialog box, we could potentially cutoff a huge number of questions from new users.
4. Question Template—We actually use this internally for bugs quite a bit at Microsoft to make sure newbies like myself are writing good bugs in our bug database—detailed templates. When a new user makes it to the post dialog box, there will already be a framework inside the text box to help them frame their question, prompting them to fill in a detailed description of what they are trying to accomplish, sample code, a list of things that they have already tried to solve the problem, and the detailed error message or behavior that they are experiencing.
5. Automatic Alert of “Bad Question” —The best way to identify a “bad question” is usually the fact that a bad question won’t get any responses in over two days. If a user hasn’t received any responses in two days, an alert could be sent to the user informing them that there hasn’t been any activity on their thread, and it might help to clarify their question.
That’s what I have for now—anybody else have any ideas? What do you think?
Comments
- Anonymous
May 22, 2006
The comment has been removed - Anonymous
May 22, 2006
hi, Joe
i guess the best way is the welcome e mail, but to be as short as possible otherwise the users will stop reading
as second option the moderators will push users read the sticky threads "how to get the optimal answer"
the best thing in all those solutions is the Question template with a small button beside it to show some help how to fill this field, something like the email address textbox's you a pattern beside it "example: john smith@somewhat.com" something like good question degree " 'newbie plz help ' = bad question" , 'textbox prolem' = weak question , 'how to set textbox background color? = good qustion" beside the search button that Christian suggested like find fast answer
i don't think bending a question for moderator approval will be a good idea, in most cases the moderator will answer the questions and you will not find a new moderators :p if you want that ,
regards
Haitham - Anonymous
May 24, 2006
As a participant in the forums I want the MSDN forums to be welcoming for new users that know nothing. This makes it difficult to implement mandatory searches and fields to fill in. They will either not post and abandon their developer dreams or fill in values that most likely will not help others to help them.
I think a short and concise message at the first post should help most new posters.
Post the the proper forum, read the forum title.
Provide as much information you can.
Specify what tools and products you are using.
This would of course appear in a red bold blinking font. :) - Anonymous
May 29, 2006
I think Andreas has a very good point there. If you make the registration process or the welcome message too long, people will not read it. It's like the manual of your new laptop, nobody ever reads it (mine is still in its original packing).
Having a moderator 'review' the question before it's posted will most likely produce a big overhead. (at least I think so. I have no idea what the ratio is between 'first time users' and 'regular visistors' is).
The automatic alert of a bad question is a good idea. You might want to go even a bit furter in this by sending another 'alert of bad message' after, lets say, 10 days (if the message was not altered after the first alert and no answers have been posted yet) and notify the user that the post will be deleted. If the user doesn't bother to log in to change the question (or to indicate that it shouldn't be deleted), I would be safe to assume that he or she isn't interested in the answer anymore. - Anonymous
May 30, 2006
We actually do have automatic alerts for an unreplied question implemented now (it just went live about a week ago.) I believe the user gets a mail after 72 hours now.
I'm still not convinced that we don't need some sort of mechanism in place to make sure that people are writing COMPLETE questions--I think I'm liking the question template idea more and more...
Is everyone completely opposed to it? - Anonymous
May 31, 2006
I am not opposed a template but my experience from the forums are that not everyone will fill them in correctly anyway.
I do not think we know if it will really help or not unless it is tried. Hopefully there will be less incorrect posts if it is implemented. Some posts will need correcting, hopefully less work than today.
I am more and more thinking it will help beginners in the forums to have a template form or maybe a wizard to post a question. They want to get help and from what other moderators has said, who are they really helping by beeing asked to provide more information.
I like the wizard approach, if they are asked questions in a logical order they most likely will be helped to post in the correct forum and provide the information needed to help them.
Wizard are really helpful for novice users to understand the flow of a procedure. With the current list of 120+ forums and more at ASP.NET forums it is not clear where to start. I am as an experienced user know I must look for the proper forum to post in, a novice has no idea if their question should go in VC# express, language, general windows form, windows form designer, windows control databinding or why not framework data access. ooh wait, it might be the SQL server forums as they want to use sql server express.
As a moderator, I would not want to review a question. I like the fast response questions are getting, even if off topic.
Sorry if double post, first try timed out. - Anonymous
June 21, 2006
Joe,
I think you are on the right track here. We're seriously thinking about the same issues. We are seriously looking at the pre-search, but along with that, looking at the title. If the title is in a set of certain phrases ("please help!" being one of them), encourage the user to provide more information. You may want to look at what NI has done, I really like their interface. http://www.ni.com/support/
Cheers,
Mark (from OCC2006)