by Floyd Rumohr
Imagine it: you’re finally able to make those home improvements you’ve always wanted. The interior decorator measures the living room using her foot and the architect uses a tape measure. How will that work out, I wonder? Inconsistencies in measurement are likely to result in mismatches — both in home repairs and during interviews.
Inconsistencies arise during the hiring process when interviewers ask different questions of different candidates, which can render the process unreliable as in the case of our home repair analogy: a foot for one measurement and a tape measure for another. The interior designer may report that the living room is 10’ wide but that measurement is only valid if her foot is consistently available!
“The only way to measure a candidate is to measure every single candidate with the same yardstick,” said Nancy Newell of Nth Degree Consulting during a panel discussion at the 2010 Society of Human Resource Management Conference. An interview question is the “yardstick” used to measure candidate responses. Newell says that each candidate should be asked the same question, or be measured by the same yardstick, if the evaluation process is going to be credible (Lorenz).
During a search for a director of development at a youth services organization in New York City, the interview design consisted of two interviews each with a different measurement scale:
Focus: Skills and Experience
Focus: Working Style and Compatibility
100 points for twelve weighted* questions
A-E grade for eight unweighted* questions
The executive director and search consultant scored responses by each candidate to twelve questions for the first interview. Three program staff awarded each candidate a letter grade for the whole of the second interview. The letter grading system was used for the second interview because it required less training time since most people are familiar with an A through E grading system from school.
Spontaneous questions arose on both sides of the interview relationship but candidates were not evaluated based on their responses to those questions because that would have altered the “yardstick” and reduced the credibility of the evaluation process. Several follow-up, clarifying, and idiosyncratic questions added fluidity to the conversation and were included among a subset of broader questions asked of all candidates.
The important thing to remember is the yardstick principle: if a question is important enough to ask of one candidate then ask it of all. Otherwise you could end up with a candidate that just doesn’t fit.
Next up: onboarding.
* Weight is the importance of each question characterized by the maximum number of points possible for a particular question. Please see title 3.7 in the Rumohr and Clarke Video and Template Library for more information.
References and recommended reading:
“Interviewing Do’s and Don’t’s: Lessons From SHRM 2010” by Mary Lorenz. The Hiring Site. July 15, 2010.