A situational judgment test (SJT) shows candidates realistic work situations and asks them to pick or rate the best response. The research says SJTs predict job performance modestly, with a validity around .26 in a large meta-analysis, and add a little beyond cognitive and personality tests. But none of the big studies was about salespeople, and in our own process an SJT, like any psychometric test, comes after the offer. Before the offer, you can get much of the same information from situational interview questions and a role play.
Why do SJTs appeal to owners?
Here's the fair part. An SJT looks like the job. "A long-time customer calls, angry about a late delivery that wasn't your fault. What do you do?" Candidates usually find that fairer than abstract puzzles, and owners like that it seems to test judgment rather than polish.
They're also easy to scale. Once a test is built, a hundred candidates can take it as easily as one.
What does the research say?
They predict, modestly. A large meta-analysis of SJTs found an average validity of .26 for job performance, across 118 validity coefficients and almost 25,000 people (McDaniel et al., 2007). An earlier meta-analysis by the same lead author reported .34 (McDaniel et al., 2001). So .26 to .34, depending on which study you use.
They add a little to other tests. McDaniel and colleagues (2007) found that SJTs have "incremental validity over cognitive ability, the Big 5, and over a composite of cognitive ability and the Big 5." That's a real finding, though the extra amount is modest.
The instructions change what you're measuring. When candidates are asked what they should do, scores lined up more with cognitive ability (.35). When asked what they would do, scores lined up more with personality traits like agreeableness, conscientiousness and emotional stability, and less with cognitive ability (.19). Validity was the same, .26, either way (McDaniel et al., 2007). So two tests that look alike can be measuring different things.
Interpersonal SJTs did better. A review that sorted SJTs by what they were trying to measure found that tests of teamwork and leadership skills had relatively high validities for overall job performance (Christian et al., 2010). For a sales role, which lives on relationships, that's encouraging.
Some items barely need the situation. One study found that for between 43% and 71% of SJT items, removing the situation description made no meaningful difference to how people scored (Krumm et al., 2015). That suggests many items measure general knowledge of how to behave well at work, rather than judgment in a specific moment.
Higher-fidelity exercises add more. In a study of 196 doctors applying for advanced training, an SJT added to the prediction from a knowledge test, and an assessment centre with live exercises added more again (Lievens & Patterson, 2011). It's not a sales sample, but the pattern is familiar: the closer the exercise gets to the real job, the more it tells you.
What's the catch for sales?
None of the large SJT studies we checked were specific to sales. The evidence is across many jobs. An SJT built for customer service or management might miss what matters in your sales role entirely, like how a rep handles a buyer who's afraid of choosing wrong.
There's also a fairness point. Paper and online SJTs carry a reading load. In one older study of 241 university students, the gap between Black and White test-takers was substantially smaller on a video version of an SJT than on the paper version, and the difference was linked to reading comprehension and to how test-takers rated the test's face validity (Chan & Schmitt, 1997). It's an old student sample, but it's a reminder that how a test is delivered can change who does well on it.
Where does an SJT sit in Ontario?
An SJT that asks what a candidate would do lines up more with personality traits, according to the research above. The Ontario Human Rights Commission's guidance says tests that assess "psychological or personality profiles" of applicants "should never be administered before a conditional offer of employment, and even then should be approached with caution" (Ontario Human Rights Commission [OHRC], 2008, section IV-6).
The Human Rights Code also says a requirement that screens out a group identified by a protected ground is discriminatory unless it's "reasonable and bona fide in the circumstances," and the group can't be accommodated without undue hardship (Human Rights Code, 1990, s. 11). A test with a heavy reading load, for a job that doesn't need one, is the kind of thing that can catch you there. And applicants selected for any assessment must be told accommodation is available on request (Integrated Accessibility Standards, O. Reg. 191/11, s. 23).
In our own process, personality testing comes after the offer, and we use it for coaching and onboarding, not to decide who gets hired. Our cognitive testing and other psychometric tests, SJTs included, come after the offer too.
How to get the benefit before the offer
Ask situational questions in a structured interview. "A customer you've worked with for three years tells you a competitor is 15% cheaper. What do you do?" Same question for every candidate, scored against a guide written in advance. Our post on structured interviews for sales roles shows how.
Turn the best situations into a role play. Watching a candidate handle the angry customer tells you more than a multiple-choice answer about it. See how to run a sales role-play interview.
Build situations from your own deals. Generic situations test generic judgment. Use the moments that actually decide wins and losses in your business.
If you buy an SJT for use after the offer, ask the right questions. Was it built for sales roles? Does it ask "should" or "would"? What's the reading level? Is there a video version? Our post on how to choose a sales assessment vendor has the full checklist.
Two things that are both true
We'll own this one for HR: SJTs feel more respectful than most tests, and that's partly why we like them. Feeling fair isn't the same as being valid for your job.
Situational judgment really does matter in sales, and SJTs carry real if modest predictive value. And for an Ontario owner hiring one or two reps, a well-built set of situational interview questions and a live role play will usually give you more, sooner, and before the offer. For owners in Guelph and across Ontario, start there.
In short
Situational judgment tests predicted job performance at about .26 in a large meta-analysis (McDaniel et al., 2007), .34 in an earlier one (McDaniel et al., 2001), and added a little beyond cognitive and personality tests. Interpersonal SJTs did relatively well (Christian et al., 2010), but the large studies weren't about sales roles. "Would do" SJTs line up more with personality, and in Ontario personality-type tests shouldn't come before a conditional offer (OHRC, 2008). Before the offer, use situational interview questions and a role play instead.
Key takeaways
SJTs averaged .26 validity for job performance across 118 validity coefficients (McDaniel et al., 2007); an earlier meta-analysis found .34 (McDaniel et al., 2001).
SJTs added incremental validity over cognitive ability and the Big Five (McDaniel et al., 2007).
"Should do" instructions lined up more with cognitive ability; "would do" instructions lined up more with personality (McDaniel et al., 2007).
For between 43% and 71% of SJT items, removing the situation made no meaningful difference to scores (Krumm et al., 2015).
Ontario's Human Rights Commission says personality-type tests should never be given before a conditional offer (OHRC, 2008).
Frequently asked questions
What is a situational judgment test?
A test that describes realistic work situations and asks candidates to choose or rate the best response. Some ask what the candidate should do; others ask what they would do.
Do situational judgment tests work for sales roles?
They predict job performance modestly across many jobs, about .26 to .34 (McDaniel et al., 2001, 2007), but the large studies weren't specific to sales. Situational interview questions and role plays built from your own deals are more directly tied to the job.
Can I use an SJT before making a job offer in Ontario?
Not in our process. All psychometric testing, SJTs included, comes after the offer. A "would do" SJT also lines up with personality traits, and the Ontario Human Rights Commission says personality-type tests should never be given before a conditional offer (OHRC, 2008). This is general information, not legal advice; confirm with employment counsel.
Are video SJTs fairer than written ones?
In one older study of students, the Black-White gap was substantially smaller on a video SJT than on paper, linked to reading demands and to how fair the test looked to candidates (Chan & Schmitt, 1997). Smaller isn't zero, and it's one study.
Work with us
Down to your last few candidates? Our Final Candidate Review tests your finalists against your own best people before you make the offer. If it doesn't help, you don't pay.
This post is general information, not legal advice. Confirm employment decisions with employment counsel.
References
Chan, D., & Schmitt, N. (1997). Video-based versus paper-and-pencil method of assessment in situational judgment tests: Subgroup differences in test performance and face validity perceptions. Journal of Applied Psychology, 82(1), 143-159. https://doi.org/10.1037/0021-9010.82.1.143
Christian, M. S., Edwards, B. D., & Bradley, J. C. (2010). Situational judgment tests: Constructs assessed and a meta-analysis of their criterion-related validities. Personnel Psychology, 63(1), 83-117. https://doi.org/10.1111/j.1744-6570.2009.01163.x
Human Rights Code, R.S.O. 1990, c. H.19, s. 11. https://www.ontario.ca/laws/statute/90h19
Integrated Accessibility Standards, O. Reg. 191/11, s. 23. https://www.ontario.ca/laws/regulation/110191
Krumm, S., Lievens, F., Hüffmeier, J., Lipnevich, A. A., Bendels, H., & Hertel, G. (2015). How "situational" is judgment in situational judgment tests? Journal of Applied Psychology, 100(2), 399-416. https://doi.org/10.1037/a0037674
Lievens, F., & Patterson, F. (2011). The validity and incremental validity of knowledge tests, low-fidelity simulations, and high-fidelity simulations for predicting job performance in advanced-level high-stakes selection. Journal of Applied Psychology, 96(5), 927-940. https://doi.org/10.1037/a0023496
McDaniel, M. A., Hartman, N. S., Whetzel, D. L., & Grubb, W. L. (2007). Situational judgment tests, response instructions, and validity: A meta-analysis. Personnel Psychology, 60(1), 63-91. https://doi.org/10.1111/j.1744-6570.2007.00065.x
McDaniel, M. A., Morgeson, F. P., Finnegan, E. B., Campion, M. A., & Braverman, E. P. (2001). Use of situational judgment tests to predict job performance: A clarification of the literature. Journal of Applied Psychology, 86(4), 730-740. https://doi.org/10.1037/0021-9010.86.4.730
Ontario Human Rights Commission. (2008). Human rights at work 2008 (3rd ed.), section IV-6: Requesting job-related sensitive information. https://www.ohrc.on.ca/en/iv-human-rights-issues-all-stages-employment/6-requesting-job-related-sensitive-information
