If I Only Could Change the Interview Process

I think Microsoft has one of the best interview processes of any major corporation around. There are other companies that use a similar approach; others, I know, use a very different approach. Despite the fact that I like our interview process, a colleague asked me a few days ago, what I’d change in our interview process. I think it’d be interesting to blog my answer, so, let me share my thoughts.

I’d like to see a few changes that may make a positive difference, and I think there are companies out there doing that. I’m not pretending to be like the great Mini-Microsoft blog; it’s just food for thought.

Gretchen Ledgard and Zoe Goldring here know everything about the interview process; they are from HR. In my case, my suggestions below are based on my personal experience and reflect just *my* perceptions.

A few facts before starting:

- Last year I badly wanted a position in another team. I was so nervous during the interview that I ended up having my worst interview ever. I couldn’t answer things that I’m used to blogging about, and I lost my concentration from the very beginning. I didn’t get the position that I really wanted.

- A quick calculation shows me my “approval rate” during my career before joining Microsoft was much better, actually, very high. In other words, it was easier for me to change companies in the past than changing teams at Microsoft nowadays.

- At the same time, my internal “approval rate” when trying to change teams is something close to 50%. Therefore, I may end up getting the position I was approved for but not the position I really wanted in the first place. Bottom line is: even if there were better candidates for the positions I applied for in the past, it may explain part of my failures to get approved. Another part, I believe, is the interview process itself, since you tend to be better in interviews for positions that you consider to be not ideal positions for you and, of course, you tend to be nervous and make mistakes when you really want a specific position. It’s psychological. Just out of curiosity, my friends have about the same “approval rate.” Since I’m already a full time employee and have gone through the interview process before, it should be easier for me to change positions and avoid losing the position I want, based on how well I was prepared for the interview. There’s a strong psychological factor affecting the performance during the interview, mainly when you’re being interviewed for THE position you want.

- It’s very easy to reject someone during the interview process even if the candidate rocks. However, it’s very difficult to select the right candidate. Even worse, the right candidate may not be approved based on the traditional interview process.

That said, what I’d like to see as part of the interview process is:

- Software matters! If the candidate has a cool utility (tool) or any other kind of software he created (maybe as a hobby), it should be considered more important than the ability to answer technical questions. For me it shows passion, and it’s the real proof the candidate created something. Questions about the software (tool, demo, utility…) should be part of the interview process. If the candidate really created the software, he/she will be able to talk about it.

- Have the candidate do your work for a few hours or something close to your daily work and monitor how he or she performs. Of course, don’t be at his/her side all the time; otherwise, it’ll affect the candidate’s performance. Give him/her a break and from time to time talk with him/her to see how things are going. Easy! The candidate should feel comfortable and have space for creativity here. This is what I’d really love seeing! Supposing the candidate has a good performance, isn’t it enough to have a good idea about his/her potential?

- Use technical questions during the first part of the interview, just to make sure you’re not going to interview the wrong candidate, but don’t use them to judge the candidate’s potential. Also, the questions shouldn’t be difficult. It’s easy to memorize things and be prepared for the interview and look good when answering questions. However, the real goal here is not to hire someone technically able to explain all C++ language constructs or all .NET Framework classes or yet someone that spent weeks studying for the interview process. We want someone who thinks outside of the box and has the ability to create and innovate. In my opinion, creativity is way more powerful than knowledge. I guess it’s difficult to accurately measure creativity, but it’s easy to recognize the lack of creativity. J

- Be flexible, your interviewee may be under stress and afraid of losing his/her dream job opportunity (I’ve been through it two or three times before): A nervous candidate may not be able to solve a logic problem or to create an algorithm under pressure during the interview. It doesn’t mean the person doesn’t have potential.

- I like puzzles. But I’ve learned they might not be an effective tool to hire people, so it’s better not to judge someone based on his ability to solve one specific puzzle.

Considering the time pressure and psychological factors, it’s better to analyze the approach the candidate takes to solve the problem, to look at the way the candidate thinks, not the solution. Besides, there’s the possibility the candidate has been studying puzzles to be able to solve them during the interview process.

- Blogs and utilities. There’re tons of blogs scattered in the internet. Some of them are pretty cool (I’m talking about content)! Again, it should be something to be considered during the interview process and not ignored. Last year I contacted two blog authors to see if they were interested to try an interview in my previous team. Needless to say, I was impressed by the content of their blogs. I’m also impressed when I see some cool free tools in the internet. In my opinion one cool tool is worth a thousand of words.

- I also think that internal candidates moving to other internal positions should have a conversation with the new manager and the team. I see no reason to go through the entire interview process over and over again. One more thing: no more permission to interview.

I believe this approach is more efficient for identifying talent, since it’s less “binary,” gives more space for creativity, and should help reduce the stress level during the interview process.

For internal candidates moving to other teams, the process should be easy and direct. It’d increase the probability of choosing the right person for a specific position at the right time, boosting the employee morale and productivity. It’s a win-win situation for the employee and for the company.

Comments

• Anonymous
  July 15, 2008
  PingBack from http://blog.a-foton.ru/2008/07/if-i-only-could-change-the-interview-process/

• Anonymous
  July 15, 2008
  I blogged about my experience with a recruiter for the Dynamics AX team last week: http://weblogs.asp.net/jezell/archive/2008/07/13/my-first-microsoft-interview.aspx I've only had the phone part so far, but I was impressed. It's not very usual that I get caught off gaurd. I'm the guy that people on my teams go to when they can't figure out what's going on, and I can usually give an quick answer after hearing the problem. However, the recruiter managed to catch me off gaurd with a couple questions, which is pretty cool.

• Anonymous
  July 16, 2008
  Hmmmm... your comment gave me an idea. I'll write about the interview process, using all public material that I have. :)

• Anonymous
  July 16, 2008
  I agree with the idea that "Software matters" and all of your logic behind it.  Unfortunately I don't think it could ever be included in every interview process because of fraud concerns.  It's really easy to present a tool someone else wrote as your own work and do a little bit of homework in order to deflect any obvious questions of fraud.     This is really unfortunate because I find that previous projects are a great way to judge a candidate.  By sheer luck over the years I've interviewed several people who did (or claimed to have done) projects in areas I specialized in college.  In the fraud cases I was able to quickly detect them because I had domain specific knowledge of the situation and was able to ask in depth questions.  Without the domain specific knowledge my ability to detect fraud wouldn't be nearly as good.   For the people truly representing their work I was able to get a great sense of their involvement and understanding of the problem space.  

• Anonymous
  July 16, 2008
  Right, but fraud may happen with puzzles or technical questions, don't you agree? A candidate can easily be prepared to answer questions he/she couldn't answer with no previous study and preparation. Thus, we would end up measuring the candidate's capacity of learning how to survive the interview process while pretending he/she had no previous preparation before. Maybe this approach of "software matters" could be used for employees when trying to move to other internal positions.

• Anonymous
  July 16, 2008
  @Roberto, Definately agree fraud can happen with any question you ask an interviewee.  However a good interviewer should be able to spot the majority cases of fraud. When an interviewer asks a question from a domain they understand then they are at a significant advantage because they are specifying the knowledge domain and problem space.  If someone can fruad their way through an interview in a domain you understand then either 1) you are a bad interviewer or 2) bad luck (candidate just happened to study that problem all night the night before the interview).  I think the latter is rare enough to not be a significant problem. If you frame an interview around a piece of software a person wrote then they control the knowledge domain and problem space.  This makes fraud on the part of the interviewee much simpler.  I just choose a domain you don't understand and learn the bare minimum to look competent.  

• Anonymous
  July 16, 2008
  Hi Jared, Great argument, I do agree with you. However, I believe it doesn't invalidate the approach when interviewing internal or external candidates that are coming from the same domain. For the interviewer it shouldn't matter, but for the candidate it's easier to discuss about something he/she created than trying to remember answers for things he/she may have forgotten. It's better to evaluate the candidate for something important that he/she had created and can show us, if it's part of the same domain from the interviewer's daily work, than evaluating the candidate for his/her ability to answer technical questions and ignore any creative work done before. We may end up hiring the guy that was prepared for the interview process and discarding the guy the has potential and creativity, but hadn't prepared himself for the interview. This is an issue for internal candidates, too, since we use the same approach.

• Anonymous
  August 06, 2008
  Cara, I agree that there are terrible interviewers (including me) but I have learn something very valuable. Hire should focus a lot in potential and not totally in techinal knowledge. Although your current job requirements might not allow you to hire somebody with great potential who lacks the technical knowledge as it will take too long to train that person and get it started. But is worst to hire somebody with a lot of technical knowledge who lacks potential or the skills for the job. A few comments on your points:

•      Software matters! - I think this really depends on which position you're applying for and the type of individual is being interviewed. There's very good technical people with lots of passion who doesn't develop any code. Also passion for technology is only a part of the values that you should be looking for, not the only one.
• I like puzzles....I went through the internal hiring training and they recommended NOT using puzzles. As it doesn't really tell you how smart is that person it might only mean that person saw that puzzle before and memorized the solution. Puzzles don't prove much.
• Have the candidate do your work.  This will only work with internal candidates as there will be too many security concerns with an external individual. Big problem with this is that interviewing takes already a lot of time. Now imagine having to train an candidate and work with him for several hours. Now multiply that for 10 candidates.

• Anonymous
  August 06, 2008
  Hey Christian! :) Your point is my point! We must hire based on potential first. It's better to hire a candidate that has a lot of potential than a candidate who is able to answer technical questions but doesn't have the same potential. The thing here is: the candidate can learn how to succeed in a technical interview and, on the other hand, a good candidate that haven't refreshed his mind and studied things we use to ask may be qualified as having less potential. This is what bothers me. Related to the three points above:

• I agree that software matters should be used for positions that requires software skills, otherwise it doesn't make sense.
• I went through the same training :) That's why I'm saying we shouldn't use puzzles anymore. Same thing that you're saying.
• Your third point is something I partially agree. It may not apply in every scenario, but I see at least one that it may work, for instance, if you are an internal candidate trying to move to another position.

• Anonymous
  August 20, 2008
  The comment has been removed