High signal job interviewing: Bayesian style

TL;DR: To get the most out of interviews, use the answers to your questions so far to guide your next question. Make your questions up on the fly. Ask yourself: “What is the question that would give me the most signal about whether this person would succeed in the role?” This approach requires practice and skill (and best applied after your first 100 interviews), but can help you get considerably more out of the interview than a fixed list of questions.

Job interviews are weird: you spend 45 minutes with someone to decide if you want to spend the next few years working with them. You may end up spending more time with that person than your spouse. That makes every minute of an interview incredibly valuable.

To deal with this challenge, and since I’m an AI person, I’ve adopted an approach based on a statistical technique: Bayesian Optimization.

The math of Bayesian Optimization is hard, but the principle is not: you are trying to work out the shape of a function f, but evaluating f(x) takes a long time, so you have to work out which value of x is the most valuable to evaluate. Bayesian optimization is a way of working out what x is, you evaluate f(x) for that x, and you repeat the process. The goal is to understand the shape of f from as few evaluations of f(x) as possible. The key thing to notice is that it’s a sequential process. Round 2 depends on what you learned in Round 1.

Applying this to an interview, there’s a person in front of you with a range of skills and abilities that you have to evaluate. You have 45 minutes, which is enough time to maybe ask 15 questions. How do you get the most out of every question?

Traditional interviewing style is that you have a standard list of questions you ask the candidate. This is not sequential the same way Bayesian interviewing is: It does not take advantage of what you are learning during the interview. It’s a good starting point, but it’s not optimal. Maybe do your first 100 interviews this way to get good at interviewing, but you’re going to hit a plateau until you learn to get more out of the interview.

Bayesian interviewing is different. Before you even start the interview:

• Consider the role. What are the most important criteria for succeeding in that role?
• You look at their resume. Anything stand out relative to the role they are interviewing for? For example, is this a role change for them? If so, a key signal is why they are changing their role. So after pleasantries, this is the first question you would ask.

If nothing stands out in the resume, you can start with something like “tell me about a recent project you worked on. What were the challenges? How did you overcome them?”

Say one of the things important for the role might be massive scale. Massive scale brings its own set of challenges. Then you can ask them what the scale of the system was, how they dealt with the issues etc.

The point is: they’ll say something in their response that relates to the most important criteria for the role. Look for it, find it, and dig into it. Say one of your concerns is technical depth. Pick a technically tricky part of their last project and go deeper until you’ve completed an assessment of their tech depth. When you’ve finished that, try to come up with the next question, on the criteria that you feel is most important.

Sometimes, this Bayesian style leads to slightly uncomfortable questions. For example, say you realize during the interview that the person has worked in mostly large tech organizations, and you are interviewing for a role in a startup. A critical question is can they adapt to a more ambiguous, DIY fast paced environment. But that could upset the interviewee: “You’ve mostly worked in slow moving large companies, what makes you think you can succeed here?” could be misinterpreted as arrogance.

In such cases it’s important to phrase your questions empathetically. Often explaining your intention can help put the interviewee at ease. In the above case for example, you might ask: “We’ve observed that sometimes the transition from a big company to a startup doesn’t go well, mainly because of the pace. Is there an example from your career where you’ve executed on a project that needed to be delivered quickly? I am asking because I’d love to hear from you so I can alleviate that concern.”

Bayesian interviewing takes a bit of practice and relies on you being able to come up with questions on the fly instead of sticking to a pre-written list. But by asking questions sequentially, using the information you’ve already learned to inform the questions you ask next, you can get considerably more signal in the same amount of time.