Technical Interviewing Part 2: Requirements

Technical Interviewing – Part 2: Requirements

This is part two of a series on technical interviewing. Reading Part 1 will give you a good introduction to technical interviews and the pitfalls many interviewers fall into. Part 2 gives advice on creating a job description that makes finding a great technical candidate much easier.

Job Description

A great technical interview starts with a great job description. Great job descriptions are hand-crafted by the people who care the most about the success of the newly hired person: the gaining supervisor and team. The supervisor should put some thought into crafting the description of the responsibilities of the new hire. The new hire’s team members should check the supervisor’s vision and give feedback. Finally, the senior managers or human resources folks should take a quick pass at the description to confirm that the technical team put together something that’s comprehensible to humans. If anything is unclear, it should be explicitly spelled out. A job description should accurately describe what the person will be doing, the environment they’ll be doing it in, and, most importantly, what they must do to succeed. A lot of thought goes into the description. The description crafter needs to convey information accurately to a candidate so that they can visualize themselves in that environment. Obscuring parts of your company or the responsibilities of a candidate is a great way to waste a lot of time interviewing and producing no results, or even worse, hiring and firing candidates that just don’t seem to fit in. Interviewing can take some time as you find the person that will spend a lot of time with you and contribute to and thrive off of the culture you’re developing. The last bit of a job description is a list of technologies the candidate will be working with. I purposefully leave this to the end. This may seem like the easiest part, but it does need some thought before you pen the list of technologies you’re using.

Expressing A Technical Skill

There are a few ways to list a skill on a job description that are clear and concise. These are all useful for different purposes. The most concise way is to just prefix the skill with a qualifier: Requires 3DS Max Experience. Another way is to have 3 blocks in the job description that show all the skills that fall into the categories of Qualifiers below.

Qualifiers

  • Requires – A candidate must have technical experience in this skill.
  • Preferred – We’d really like a candidate to have experience in this skill.
  • None – Leaving of a prefix is just a way of saying “we use this technology”.

As a rule, prefer the less restrictive qualifiers unless you have very good reason to have a preference or a requirement. Why? Because most people see a requirement and back off if they don’t feel confident by their own self-measure, and that’s not a good thing. A lot of folks I’ve talked to are okay with using high requirements as a gate to weed out potential interviews. The problem with that is that you’re being vague. You’re enforcing your idea of what a required level of skill is without communicating that explicitly to the candidates. The person that evaluates their skills more humbly than you do might be the perfect fit for your team, but they won’t even apply because they don’t think of themselves as “ninja”, “guru”, or “master” of a skill. Some of you might also be asking, “What’s a good number of years of experience to put on these skills?” or “Where do the degree requirements go?”. To those folks I say, “Nowhere”. Talent isn’t measured in years of experience, degrees, or credits. Just look at the game programmer applicant with 7 years of experience in the industry who’s done nothing but UI scripting who wants to move on to AI development. He’s got the 7 years in C++ on his résumé and experience closing on 3 shipped AAA titles, but he’s fallen into the paradigm of UI scripting. Chances are he’ll apply while the guy with 1 year of indie development experience that has killer insight into AI processes is going to pass on your position that requires 2 shipped AAA titles, 5 years of industry experience using C++, and a doctorate in Applied Artificial Intelligence.

Common Cases

Take a breath and think about what type of person you want to have on your team. I’m going to generalize here for the sake of example. Below are a few common cases where you might need to hire a technical person. These cases need to represent the required skills in a completely different way to the prospective team members.

Back-Fill

The team that’s mid-project and needs to back-fill a position can’t just snatch up a body from the street that has all the right buzz words on their résumé. These teams still need to care about their company’s culture and the people they allow into their team. That means a “quick” back-fill might take longer than you’d like. A technically competent person that joins a team and siphons efficiency and destroys team cohesion will be much more costly than a person that’s a great fit that joins later. I’d rather move a deadline than destroy a team. The skills of these quick-joiners will be evaluated based on their skill with the technology the team is using. What you need at this point is someone who can come into a project and immediately pick up the slack left by a missing team member or add some velocity to a lagging project. This is the only case where I recommending listing a “Requirement” for specific technologies. If you’re making a JavaScript game (WHY!?) and you have a 3 month deadline, you need someone who knows JavaScript. There isn’t time to train someone who may be an excellent overall developer in the finer points of JavaScript game development.

The Junior Candidate

The junior candidate positions exist somewhere between the extreme position above and the general rule below. Junior candidates are great to talk to when you’re considering the future of your company or team. Junior folks generally have a lower price tag and can be more easily molded into your culture and technical standards. They most likely haven’t had time to learn great technical truths yet though. They either have the basics they’ve been taught in college, which are difficult to ascertain from college to college, or they’re self-taught technical people who are driven enough to solve the problems they’ve faced and may have holes in their general domain knowledge. When crafting technology listings for these types of positions the relevant technology should be listed listed as “Preferred”. When hiring a junior candidate you know that there will be mentoring required. You know that the innate knowledge of your technology may or may not be there. The uncertainty of the quality of knowledge these candidates have with a particular technology is okay. What you really want from these candidates is not their technology experience, it’s their raw talent. You get to cultivate a talented person and give them the skills you need for both of you to succeed. Competency will be tested, so don’t think that junior developers are exempt from analysis, but you don’t want to try to find someone with the knowledge of 5 years of experience with Photoshop for a junior position – you want an intelligent person that can learn and that is flexible enough to use wherever technical skill you require of them.

The Subject Matter Expert

Sometimes you really need someone who has experience in a technology or specific set of domain knowledge. If you’re making games and you need an engine programmer to lead your team, go ahead and add “Experience with engine Development” as a requirement. Just don’t require 5 shipped titles, post-graduate work in AI and 20 years of experience. Let people surprise you with their knowledge. Test for technical knowledge, don’t filter on time-in-domain.

The Permanent Hire

Now that the exceptions are out of the way, let’s get to the rule. I don’t want to see someone hiring a full-time long-term developer and requiring a specific language or an artist requiring a specific set of tools. I want to see what tools and technologies you’re using in the job description, but it better be at most a “Preference” and not a “Requirement”. Why? Because you don’t really want a Python developer even though you’re developing in Python. You don’t want a Photoshop artist even though you’re creating content with Photoshop. You want someone who views tools or languages as a set of clothes that are worn for particular occasions. The people who are able to change those “clothes” more frequently than a character in Downton Abbey are the ones that have true skill and talent at their core art. They understand the concepts and problems that the tools solve. What you really want is a great skilled candidate, and that’s the point of this article. A competent developer/artist/technical person is going to easily switch from one tool to another. Now, make sure that you do indicate all the technologies and tools that you use. It’s frustrating for a person that only likes to develop in C# to get through an interview only to find out that the company only develops in VB.Net.

Traits

Traits are different from the skills identified in the previous section. A skill can be acquired by simply reading a book or taking a class and then developed over time. Traits are much more difficult to acquire. These are generally developed over time with experience. That’s not to say that junior people lack traits. A passionate person can invest the time necessary to develop a trait. Since traits are difficult to train, these are much more important than skills, which can be taught easily. If you want to hire quality people and build a world-class team, you’ll hire people for their traits and mold their skills. Traits may or may not make it to a job description, but the traits you find relevant to the position must be evaluated. These Here are some examples of critical traits for technical people of various types:

  • Willing Learner
  • Passionate Technologist
  • Clear Communicator
  • Critical Thinker
  • Abstract Thinker
  • Plays well with Others
  • Spatial Awareness
  • Calm Under Pressure

Technical Interviewing: Part 3

This part of the series has been all about the job description which will prepare both you and prospective candidates for part 3: The Technical Interview Process. Part 3 lays out a structure for how to handle the process of funneling applicants through a pipeline to the interview table.

Resources

EntreLeadership: 20 Years of Practical Business Wisdom from the Trenches

TechnicalInterviewingPart1

Technical Interviewing – Part 1: Pitfalls

Technical Interviews

Anyone who’s conducted an interview can tell you that interviewing any candidate is difficult. The process can become even more difficult when conducting technical interviews. Technical interview difficulty is compounded by having the wrong interviewer to find your candidate or by having weak or unclear expectations about the requirements for the candidate. Perhaps you are a non-technical person interviewing someone for a technical position, or a technically skilled person conducting an interview for a position with a different knowledge domain (like a chemist trying to hire a data analyst). Maybe you have the technical skills for your domain, but you lack the valuable skill set required of a good interviewer. None of these situations are easy to overcome, but hopefully I can share some of my experiences to help improve the process of matching qualified candidates to desirable positions. First, a word of caution to job seekers that might be reading this series: This is how hiring should be. You should not expect potential employers to follow this process, but if they do, you’ll know that you’ve found a quality company. Also, this is generally written from the point of view of hiring a programmer, but the concepts can be abstracted and applied to particular technical domains.

The Purpose

Obviously the aim of interviews are to evaluate the candidate’s ability to carry out a job, but interviews also allow the candidate to understand the company from a perspective they are unable to get outside of the company. Initial interviews should handle the personality fit of a candidate, establish a mutual excitement (or at least interest) for both interviewer and candidate, and make sure that the candidate’s vision for the future and the goals and mission of the company are in alignment. Often a technical interview is one of the first face-to-face interviews conducted. From the interviewer’s perspective the technical interview should yield enough knowledge for the interviewer to say either “Yes, this person meets the technical requirements of the position”, or “No, this person isn’t able to do the required job”. More importantly, the interviewer should have many qualitative observations of candidates to allow them to find the best fit for the company from a field of candidates.

The Pitfalls

Non-Technical Interviewer

The non-technical person, like the owner of a small business hiring their first web developer, might merely rely on a list of questions found on the internet to throw at a candidate. The risk here is that the candidate might have searched the internet and found the same list of questions during interview prep. This only yields the information that the candidate does interview prep work, has strong Google-Fu, and probably doesn’t give the narrative of the candidate the interviewer is seeking. I would always encourage a non-technical person to have a “wing-man” during a technical interview that has at least a basic level of understanding of the technical issues being discussed to make sure that a candidate isn’t just blowing smoke and taking advantage of a non-technical interviewer.

Cross-Domain Technician

Quite often an organization will need to hire someone with similar skills to other technical people on their team, but dedicated to a different domain of applied knowledge. A great example is a game development company that needs to hire a web developer. There are many aspects of programming that are common to all developers, but there’s a huge difference in the experiences of a web developer and a game developer. A game programmer interviewing a web developer might falsely expect that a certain level of knowledge in a particular technique is required in a web development position and discount a person that is highly qualified in their domain due to a lack of knowledge in the interviewer’s own domain.

Subject Matter Expert

The opposite end of the spectrum from the non-technical interviewer is the small software engineering team that’s expanding to add another developer. They use their exceptionally knowledgeable team member to conduct the interview. Without interviewing skills these interviews generally end up making the candidate feel like they’re subjected to a haughty comparison or an inquisition as the inexperienced interviewer throws obscure technical questions at the candidate. These only serve to find people who revel in the obscure, not necessarily find quality candidates.

Traps

Here are some common traps that interviewers fall into when it comes to technical interviewing.

Quiz-Style Questions

Asking quiz-style questions tests the candidate’s ability to retain information and parrot it back – not exactly what defines a successful candidate. Examples include things like “Describe the Model-View-Controller pattern,” or “When is the OnAwake event fired?”. These questions are helpful, but not in the way that most people think. The place for these questions is in a questionnaire. A good interview tactic is to send out a “written test” before a technical interview. These quiz style questions can be on that test and don’t necessarily serve to exclude a candidate, but their responses should become points of discussion in the technical interview. The outcome of discussion of responses to quiz-style questions is much more valuable than a correct answer to a question.

Obscure Knowledge Questions

These types of questions are useful to explore the extent of a candidate’s knowledge, but should never be used to exclude a candidate. The thing you’re testing with obscure knowledge questions is generally experience, and even worse, specialized experience. It’s much more valuable to have someone who is an expert problem solver than someone who has solved a particular problem. These types of questions will favor the person that has solved the problem and not the expert problem solver.

A Bad Example

I was recently interviewed for a full-time C# developer position with a company that creates complex solutions that I imagine requires a good measure of technical competence to create. My interview was with two members of the development team. They fell into the first trap – asking nothing but quiz-style questions. The technical part of my interview involved questions like:

  • “Describe some programming patterns.”
  • “Define overriding and overloading.”
  • “What is an abstract class?”

From my responses to this evaluation I was pronounced to be “very knowledgeable” about C#. Now, there’s no problem with that final evaluation – I am very knowledgeable about C# – but that statement is not supported by the results of these questions. Sure, I gave them more than they actually asked for. I defined 4 programming patterns and gave examples of their implementation in recent projects (and I didn’t even pull out the Singleton pattern!). I not only defined overriding and overloading, but gave the conditions in which I have and would use each. I also illustrated when I used abstract classes in C++ and when and why I might use interfaces in C# instead of abstract classes. But for most candidates who don’t understand that in interviews you never merely give the answer to the question but also provide examples to show competency and experience, defining these terms would have given absolutely no insight into the candidate’s technical ability at all, just their memorization or searching ability.

A Better Example

Lets play pretend. Lets say that they wanted to find someone to add to their team for the long-term that could develop internally used web applications using C#. What should they have asked in the technical interview? The questions I was asked in the above example are perfectly fine light technical questions and are a great lead in to real technical discussions or as a pre-technical screening discussion. Here’s what I would have rather seen in this situation.

  • What components make up a great programming interface? Why?
  • Use the whiteboard to show a diagram for a particular system. Why did you choose this architecture/pattern?
  • Describe a system that solves a particular problem. Now what would change in your design with a change in requirements?
  • Solve a problem outside of your general domain of knowledge

These questions get all the answers to the previous questions and test the quality of any developer for any language. These questions don’t ask anything that’s particular to a knowledge domain within programming. These questions are all open questions that allow technical people to explain themselves to the interviewer and expose the thought processes of the technical candidate. The best part of these questions, especially when interviewers specify that the interviewee should think out loud and ask questions of the interviewer (“Use us as if we are Google”), is that they prompt two-way communication between the interviewer and the candidate allowing the candidate can ask questions of the interviewers and the interviewers can give feedback to the interviewee. These types of interviews allow candidates to relax a bit more (usually), expose their thoughts (a critical piece of information for an interviewer), and give them the opportunity to “wow” the interviewer by demonstrating unexpected solutions or creative thinking.

Comparison

The candidate that makes it through the bad example has demonstrated:

  • The ability to define keywords

The candidate that makes it through the better example has demonstrated:

  • Understanding of general technical concepts (in this case programming concepts).
  • Their level of comfort presenting technical information.
  • Their level of technical ability.
  • The types of questions they ask to gain information.
  • Their understanding of how technical systems build off each other and interconnect.
  • How they approach problems for which they most likely have no existing knowledge.
  • Shows how they handle constructive discussion of their work.

It’s pretty obvious which results in a better source of information. You have much more information about a candidate that undergoes the second type of questions. This data is useful for the types of interviewer situations described at the beginning of this article. The non-technical interviewer gets to find a candidate that’s able to speak on the interviewer’s terms about technical concepts. The cross-domain interviewer gets to leverage their own technical knowledge to find commonality to validate candidate traits. The subject matter expert interviewer gets to ask follow-up questions to satisfy their level of curiosity and help prevent interview from degrading into an interrogation.

Next

Part 1 has been an introduction to the challenges of interviewing technical candidates. The next section of this series discusses the first step fixing these problems. The solution starts before the interviewing even begins, with a great job description. Keep an eye out for Part 2: Requirements.

Resources

EntreLeadership: 20 Years of Practical Business Wisdom from the Trenches