malvasia bianca

One of the least pleasant aspects of hiring is rejecting candidates. (More actively unpleasant for them than for me, to be sure.) It’s something which, until recently, I did almost exclusively over e-mail.

Sometimes, rejection over e-mail makes sense. I typically put candidates through up to three stages of filters. (Not counting the initial resume screen.) The first stage is a sanity check over the phone: is there some obvious reason why this situation is a misfit that I didn’t figure out from the resume? Almost everybody makes it through this stage; in the few places where a candidate doesn’t make it through this stage, it’s because it’s clear to both of us that their goals aren’t a fit for this position, so we agree that it’s not a match, and it’s simply not a matter of me rejecting them. (One could argue that I should make my phone interview stricter and reject more candidates at this stage; for better or for worse, I’m not doing that yet, and it’s tangential to the issues in this post.) The third stage is a half-day team interview; in this situation, my team discusses the candidate as a group after the interview is over (usually the next day), so I’m simply not in a position to deliver an immediate verdict to the candidate.

The second stage is the tricky one. This is a one hour in-person interview, where I ask some questions and have the candidate do a bit of programming. Sometimes, after the end of the interview, I’m genuinely not sure whether or not I want to bring the candidate back. More frequently, however, I am sure, and the answer is “no”. I can’t remember having second thoughts about my initial no reaction at this stage, so why not deliver the news then?

I think my actions started to bother me because of some blog posts that I’ve read recently, but I just looked at the usual suspects, and I couldn’t find any relevant posts. Certainly one reason why I’m thinking about this, though, is that I’m in the middle of rereading Gerald Weinberg’s Congruent Action. I can’t claim that I’m acting congruently in this case: I don’t believe that it makes job candidates any happier to have to wait and wonder for a day or two. Nobody likes being rejected, but you might as well get the news as soon as it’s available. The only reason why I’m delaying is to avoid in-person awkwardness; it’s just a sign of lack of courage on my part, with no obvious benefit to anybody.

So today I rejected two candidates in person. And, actually, it turned out rather well: both were disappointed, but both took it well, and both asked for advice about what they could have done better. In retrospect, it seems that I haven’t been saving myself any grief: all I’ve been doing is elimimating a potential learning opportunity for the job candidates. (Which raises the question: what learning opportunities might my actions in this regard be costing me?) Nice to have immediate positive reinforcement like that: I will try to continue to behave this way in the future. (And I should spend some time thinking about how to politely reject people in person.)

To be sure, I don’t claim to have any great pearls of wisdom on what the candidates could have done better. But, in time-hallowed blogger tradition, I won’t let that stop me from sharing my thoughts on the subject. The interesting thing about this case was that both candidates failed for the same reason: neither was completely hopeless, their initial attempts at a solution were both reasonable, but both floundered significantly when debugging. (Learning about people’s debugging skills is actually my goal in the programming question: I’m always a little disappointed when people solve the problem without making a mistake, because I know that, no matter whom I hire, they’ll make programming mistakes with some regularity, and I want to see how they deal with that.)

And, in both cases, it seemed like they didn’t know what they were looking for when debugging, or indeed when testing the program before discovering that debugging is warranted. (I tell them to implement a function and do enough testing to convince themselves that it was correct.) I suspect that both of them could be helped by taking a more scientific approach to debugging.

Sometimes, scientists are just gathering data: poking around, seeing if they see anything interesting. But science really progresses when people are making and testing hypotheses. Make a prediction that is concrete enough to be testable, to be falsifiable, and then test it. If the test results match your prediction, that’s always fun; if not, though, you’ve still learned something concrete, and can use that to make further hypotheses.

And this works for debugging, too. At the very basic level: if you think something is wrong with your code, you should know what you expect to have happened, so you can tell whether or not something really did go wrong! But it helps to make concrete predictions at a more refined level than that: “we know that something went wrong overall because we saw X instead of Y. If the problem isn’t in this part of the code, then expression E should have value V, whereas if it is there, then E should have a different value”. If you do this a few times, you should be able to zero in on the problem quite quickly, much faster than you could have by just looking at the code and hoping for inspiration.

This helped crystallize one thing that I think is wrong with printf debugging. Many people’s first reaction, when confronted with misbehaving code, is to sprinkle printouts throughout the code and hope that enlightenment results. This can be great, if you have specific hypotheses about what the output of those messages should be. I think that many people, though, don’t really have specific hypotheses in mind, just a vague feeling of values that they’re interested in. When this happens, printf debugging doesn’t lead to answers very quickly: the messages give you a lot of data, data that can easily lead you down all sorts of unproductive paths, data that can fool you into thinking that you’re learning something when you don’t really know what that data represents. (I suspect that debugging with a debugger is less likely to lead to this problem, if only because it takes more effort to generate lots of values, so you’re more likely to spend time thinking about what values you want to generate.)

Maybe I’m unfairly characterizing printf debugging, but I will stand by the value of concrete hypotheses when debugging. Which raises the question: if it’s so good when debugging, can we use the same ideas when writing new code, code which we don’t yet have reason to believe is incorrect? The answer is yes, of course: that’s exactly what test-driven development is all about.