TechnicalInterviewingPart1

Technical Interviewing – Part 1: Pitfalls

Technical Interviews

Anyone who’s conducted an interview can tell you that interviewing any candidate is difficult. The process can become even more difficult when conducting technical interviews. Technical interview difficulty is compounded by having the wrong interviewer to find your candidate or by having weak or unclear expectations about the requirements for the candidate. Perhaps you are a non-technical person interviewing someone for a technical position, or a technically skilled person conducting an interview for a position with a different knowledge domain (like a chemist trying to hire a data analyst). Maybe you have the technical skills for your domain, but you lack the valuable skill set required of a good interviewer. None of these situations are easy to overcome, but hopefully I can share some of my experiences to help improve the process of matching qualified candidates to desirable positions. First, a word of caution to job seekers that might be reading this series: This is how hiring should be. You should not expect potential employers to follow this process, but if they do, you’ll know that you’ve found a quality company. Also, this is generally written from the point of view of hiring a programmer, but the concepts can be abstracted and applied to particular technical domains.

The Purpose

Obviously the aim of interviews are to evaluate the candidate’s ability to carry out a job, but interviews also allow the candidate to understand the company from a perspective they are unable to get outside of the company. Initial interviews should handle the personality fit of a candidate, establish a mutual excitement (or at least interest) for both interviewer and candidate, and make sure that the candidate’s vision for the future and the goals and mission of the company are in alignment. Often a technical interview is one of the first face-to-face interviews conducted. From the interviewer’s perspective the technical interview should yield enough knowledge for the interviewer to say either “Yes, this person meets the technical requirements of the position”, or “No, this person isn’t able to do the required job”. More importantly, the interviewer should have many qualitative observations of candidates to allow them to find the best fit for the company from a field of candidates.

The Pitfalls

Non-Technical Interviewer

The non-technical person, like the owner of a small business hiring their first web developer, might merely rely on a list of questions found on the internet to throw at a candidate. The risk here is that the candidate might have searched the internet and found the same list of questions during interview prep. This only yields the information that the candidate does interview prep work, has strong Google-Fu, and probably doesn’t give the narrative of the candidate the interviewer is seeking. I would always encourage a non-technical person to have a “wing-man” during a technical interview that has at least a basic level of understanding of the technical issues being discussed to make sure that a candidate isn’t just blowing smoke and taking advantage of a non-technical interviewer.

Cross-Domain Technician

Quite often an organization will need to hire someone with similar skills to other technical people on their team, but dedicated to a different domain of applied knowledge. A great example is a game development company that needs to hire a web developer. There are many aspects of programming that are common to all developers, but there’s a huge difference in the experiences of a web developer and a game developer. A game programmer interviewing a web developer might falsely expect that a certain level of knowledge in a particular technique is required in a web development position and discount a person that is highly qualified in their domain due to a lack of knowledge in the interviewer’s own domain.

Subject Matter Expert

The opposite end of the spectrum from the non-technical interviewer is the small software engineering team that’s expanding to add another developer. They use their exceptionally knowledgeable team member to conduct the interview. Without interviewing skills these interviews generally end up making the candidate feel like they’re subjected to a haughty comparison or an inquisition as the inexperienced interviewer throws obscure technical questions at the candidate. These only serve to find people who revel in the obscure, not necessarily find quality candidates.

Traps

Here are some common traps that interviewers fall into when it comes to technical interviewing.

Quiz-Style Questions

Asking quiz-style questions tests the candidate’s ability to retain information and parrot it back – not exactly what defines a successful candidate. Examples include things like “Describe the Model-View-Controller pattern,” or “When is the OnAwake event fired?”. These questions are helpful, but not in the way that most people think. The place for these questions is in a questionnaire. A good interview tactic is to send out a “written test” before a technical interview. These quiz style questions can be on that test and don’t necessarily serve to exclude a candidate, but their responses should become points of discussion in the technical interview. The outcome of discussion of responses to quiz-style questions is much more valuable than a correct answer to a question.

Obscure Knowledge Questions

These types of questions are useful to explore the extent of a candidate’s knowledge, but should never be used to exclude a candidate. The thing you’re testing with obscure knowledge questions is generally experience, and even worse, specialized experience. It’s much more valuable to have someone who is an expert problem solver than someone who has solved a particular problem. These types of questions will favor the person that has solved the problem and not the expert problem solver.

A Bad Example

I was recently interviewed for a full-time C# developer position with a company that creates complex solutions that I imagine requires a good measure of technical competence to create. My interview was with two members of the development team. They fell into the first trap – asking nothing but quiz-style questions. The technical part of my interview involved questions like:

  • “Describe some programming patterns.”
  • “Define overriding and overloading.”
  • “What is an abstract class?”

From my responses to this evaluation I was pronounced to be “very knowledgeable” about C#. Now, there’s no problem with that final evaluation – I am very knowledgeable about C# – but that statement is not supported by the results of these questions. Sure, I gave them more than they actually asked for. I defined 4 programming patterns and gave examples of their implementation in recent projects (and I didn’t even pull out the Singleton pattern!). I not only defined overriding and overloading, but gave the conditions in which I have and would use each. I also illustrated when I used abstract classes in C++ and when and why I might use interfaces in C# instead of abstract classes. But for most candidates who don’t understand that in interviews you never merely give the answer to the question but also provide examples to show competency and experience, defining these terms would have given absolutely no insight into the candidate’s technical ability at all, just their memorization or searching ability.

A Better Example

Lets play pretend. Lets say that they wanted to find someone to add to their team for the long-term that could develop internally used web applications using C#. What should they have asked in the technical interview? The questions I was asked in the above example are perfectly fine light technical questions and are a great lead in to real technical discussions or as a pre-technical screening discussion. Here’s what I would have rather seen in this situation.

  • What components make up a great programming interface? Why?
  • Use the whiteboard to show a diagram for a particular system. Why did you choose this architecture/pattern?
  • Describe a system that solves a particular problem. Now what would change in your design with a change in requirements?
  • Solve a problem outside of your general domain of knowledge

These questions get all the answers to the previous questions and test the quality of any developer for any language. These questions don’t ask anything that’s particular to a knowledge domain within programming. These questions are all open questions that allow technical people to explain themselves to the interviewer and expose the thought processes of the technical candidate. The best part of these questions, especially when interviewers specify that the interviewee should think out loud and ask questions of the interviewer (“Use us as if we are Google”), is that they prompt two-way communication between the interviewer and the candidate allowing the candidate can ask questions of the interviewers and the interviewers can give feedback to the interviewee. These types of interviews allow candidates to relax a bit more (usually), expose their thoughts (a critical piece of information for an interviewer), and give them the opportunity to “wow” the interviewer by demonstrating unexpected solutions or creative thinking.

Comparison

The candidate that makes it through the bad example has demonstrated:

  • The ability to define keywords

The candidate that makes it through the better example has demonstrated:

  • Understanding of general technical concepts (in this case programming concepts).
  • Their level of comfort presenting technical information.
  • Their level of technical ability.
  • The types of questions they ask to gain information.
  • Their understanding of how technical systems build off each other and interconnect.
  • How they approach problems for which they most likely have no existing knowledge.
  • Shows how they handle constructive discussion of their work.

It’s pretty obvious which results in a better source of information. You have much more information about a candidate that undergoes the second type of questions. This data is useful for the types of interviewer situations described at the beginning of this article. The non-technical interviewer gets to find a candidate that’s able to speak on the interviewer’s terms about technical concepts. The cross-domain interviewer gets to leverage their own technical knowledge to find commonality to validate candidate traits. The subject matter expert interviewer gets to ask follow-up questions to satisfy their level of curiosity and help prevent interview from degrading into an interrogation.

Next

Part 1 has been an introduction to the challenges of interviewing technical candidates. The next section of this series discusses the first step fixing these problems. The solution starts before the interviewing even begins, with a great job description. Keep an eye out for Part 2: Requirements.

Resources

EntreLeadership: 20 Years of Practical Business Wisdom from the Trenches