Moishe Lettvin

I like coffee, bicycles, camera and code. Backend engineer at daily.co.

I Don't Know

23 February 2016

As I’ve mentioned before, my co-worker Tim & I teach an interview training class at Etsy. The 2nd most important thing we teach (after “respect the candidate”) is the concept of “mapping the potato” – discovering the boundary between what a candidate knows and doesn’t know, along many many different vectors.

Mapping a candidate’s knowledge is completely dependent upon reaching a moment when the candidate says – or implies – “I don’t know.” Getting to that state is the only way you can discover those boundaries.

I’ve always thought about this moment as the moment when the search space you’re traveling along explodes in multiple different dimensions. What you’re exploring changes from something like “show me your knowledge about how TCP works” to what can be summarized as “show me how you act when you don’t know the answer to something.” The latter is comprised of things like asking questions, applying knowledge about how computers work, communicating abstract and maybe not-fully-formed ideas, bouncing hypotheses around, dealing with ambiguity, probing for knowledge, and on and on – all of these are objectively important skills and giving a candidate a space to show them is vital to hiring well. The rate at which you, as an interviewer, are gathering data increases geometrically at the “I don’t know” moment.

But: I recently learned that my thoughts on the above lacked some depth.

This past weekend, I attended a conference with about 40 friends and friends-of-friends. We talked about all sorts of stuff, including a session on building diverse teams. Naturally, part of this conversation was about hiring (though let me take this moment to emphasize that hiring is a small part of building diverse teams).

This session gave me a chance to re-think the “I don’t know” moment. In short, and as frequently happens, I hadn’t thought in enough depth about my “white dude” privilege in this context.

Growing up in the 80s, during the era of computer “whiz kids” it was incredibly easy for me to identify with people like Bill Gates and Nolan Bushnell and Steve Wozniak and Bill Atkinson in no small part because they looked like slightly modified, older versions of me. Beyond that, I was told explicitly and implicitly – by living in a rich suburban town, and having family members who were nerdy and smart and academic, and watching movies like Real Genius that were full of nerds who looked like slightly older versions of me – that it was safe to be a little rebellious and a little weird and that even not knowing things or being selectively lazy was a form of social cachet.

Because of that, when I was personally confronted with moments as a candidate where I didn’t know the answer to something, it came naturally to me to admit that and start dissecting a problem. Don’t get me wrong – it was fucking terrifying to do this. Indeed, just last week I was the candidate in a mock interview as part of a class, and I knew the question I’d be asked, and I’m friends with the guy who was mock-interviewing me, and I even had a good sense of how I could guide the the interview down a path where the stuff-I-didn’t-know was relatively unthreatening, and I was still moderately terrified beforehand and a little shaky and had a little flop-sweat going on.

But: in the mock interview, I knew I wouldn’t be fired. And in every real interview I’ve ever done, I’ve always known I had a safety net of n other tech companies who would probably hire me. And even more than that: I knew, because everyone told me so that I did know what was up with computers. I magically get the privilege of having internalized “people who look like me are allowed to fail, so failing-to-know-the-answer isn’t the same as failing-the-interview.” I don’t have to struggle against growing up with people telling me I’m the “other” and I don’t have to struggle against whatever unconscious biases the interviewer has; indeed, the interviewer’s unconscious biases almost certainly unfairly help me. So the worst “I don’t know” moment I’ve ever experienced is qualitatively different for me than it is for somebody who’s grown up without the giant scaffolding that I get propped up by.

I had never imagined, really, what it might be like to grow up in a situation where your ability to be good at writing computer programs would be doubted because of how you looked and/or where you lived. I’d never thought how it might feel, when those doubts are constant and implicit, to say “I don’t know.”

It turns out this is a documented and named phenomenon called stereotype threat. If a candidate is part of a group that is stereotyped at being bad at something, or an outsider at a thing, the candidate is expending mental energy to counteract that stereotype, whether they subscribe to it or not. This is an insidious result of real and profound differences in what it’s like to grow up as someone who pattern-matches “someone who’s good at tech” (eg. me, a white guy who could probably fool Paul Graham) vs. being someone who doesn’t match that pattern.

So how can we counteract this during interviews?

One way is to be very explicit with the candidate that the whole point of the interview is to get to an “I don’t know” moment. Emphasize that – in this room, in this tiny little artificial thing you’re constructing with the candidate – saying “I don’t know, but I think …” is success, not failure. Repeat that at every opportunity. Admit it when you don’t know something. The interviewer’s role in the interview is not to pounce on the candidate’s failures; the interviewer’s role is to give the candidate a chance to show off their strengths – make sure the candidate knows this.

We can even go further: maybe it’s easier for candidates to work discover solutions without someone looking over their shoulders. We should give candidates space to solve problems alone, if we sense they need it – sometimes this can be a matter of leaving them alone for 10 or 15 minutes, or maybe it can be a take-home problem, or maybe set up an interview where you collaborate online with Slack.

And again: be explicit about what you’re looking for, in the spirit of helping the candidate show their strengths. If you can, ask the candidate how they’d like to demonstrate it! Would they prefer homework, or being left alone to code, or collaborating online? How do they feel they work best, and how can they best show their skills?

The more creative we can be about how we interview people, the more we can extract the signal from the noise and the more bigger chance we give everyone to succeed.

And if you’re privileged like me, keep telling yourself “I don’t know what other people’s experiences are, and I should listen.”

Thanks to Erica Joy, Marco Rogers, Alicia Liu, Camille Fournier, Peter Seibel and Brad Greenlee for their help with this piece, both by contributing to the discussion and suggesting changes. And *huge* thanks to Marc Hedlund for organizing an amazing weekend.