“If you could use just one type of assessment when evaluating candidates, what would you use?” This is a question I often ask training course delegates as a way of starting a conversation about the effectiveness of different selection methods. By far the most common response is ‘an interview’. My answer: ‘a test of cognitive ability’. Delegates are often uncomfortable with this idea despite the training courses they are on showing just how effective psychometric tests of cognitive ability are.
The evidence
Here’s a brief summary of the evidence that supports cognitive ability. In a recent comparison of 31 methods of predictive job performance, cognitive ability was by far the best predictor of both job performance and job-related training outcomes
[1]. It has been known for some time that the more complex a job the better cognitive ability predicts performance, but even for unskilled jobs it still has substantial predictive validity
[2]. The effect of ability on job performance also appears to be linear
[3]. This contradicts the view that it’s important, but only up to a point, after which we need to look for other factors – such as emotional intelligence or ‘fit’ – to identify superior performance. Whilst not discounting the potential of other factors to add to our understanding, on average, the better someone scores on a test of cognitive ability, the better they will do in the job.
How organisations can measure cognitive ability
Most tests of cognitive ability involve problem solving, either through tests of verbal reasoning, numerical reasoning or abstract / non-verbal reasoning. The complexity of problems vary from the very challenging through to simple tasks that almost anyone can get right, meaning performance depends on speed of working. Each task assesses something unique, though performance on all tests are substantially influenced by ‘
g’ or a general ability factor
[4]. General ability underpins our ability to learn. Research on children shows this association between ability and subsequent learning
[5], and is a key justification for using academic selection tests. Similarly, in work contexts, cognitive ability predicts both the rate of job knowledge acquisition and the depth of learning
[6].
So, where’s the catch?
Understanding about a candidate’s cognitive ability tells us a lot about their potential, but a few words of caution are necessary. Tests based on verbal and numerical content may be seen as work-relevant (‘face valid’) by candidates, though non-verbal or abstract tasks less so. If used, the relevance of such tests needs to be clearly explained to ensure candidates are engaged. Completing cognitive tests can be demanding, resulting in higher levels of candidate drop-off compared to other types of assessment. User experience is not always great, though some cognitive tests are now using elements of gamification as a way of enhancing experience. They can also be practiced, so it’s important to consider how candidates are prepared for these tests to ensure all have an equal opportunity to show their abilities. Many test publishers provide practice materials as a way of reducing the effect of prior experience.
Perhaps the biggest concern when using tests of cognitive ability is the risk of adverse impact, where one group of test takers performs, on average, better than another. Diversity is one of key targets for many organisations, so any assessment that threatens this can be a barrier to use. Establishing the validity of an assessment for a role means its use in selection is justified, but to many recruiting a diverse workforce is of equal if not greater importance. The causes of group differences on cognitive ability measures are complex and not fully understood, though rarely due to obvious test bias. As there is no simple way of compensating for group score differences, cognitive tests should be applied only after careful consideration as to how scores from them will be used in decision-making.
Concluding thoughts
The ability of cognitive tests to predict work performance and learning is now established. As work changes at an ever-increasing rate, recruiting people with the potential to learn and adapt to these changes is essential. Though easy to use and an effective screening tool, especially in large-scale recruitment, we need to be mindful of how cognitive tests are used if we want to create a more diverse workforce. Look out for our next blog where we will talk about what you can do to reduce adverse impact of cognitive ability tests.
Would you like to see the range of cognitive ability tests we are promoting? Click
here.
[1] Schmidt, Oh and Shaffer, (2006). The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 100 Years of Research Findings.
[2] Salgado and Moscoso, (2019). Meta-Analysis of the Validity of General Mental Ability for Five Performance Criteria: Hunter and Hunter (1984) Revisited.
[3] Coward and Sackett, (1990). Linearity in ability-performance relationships: a reconfirmation.
[4] Carroll, (1993) Human Cognitive Abilities.
[5] Deary, Strand, Smith and Fernandes, (2007) Intelligence and Educational Achievement.
[6] Hunter and Schmidt (1996). Intelligence and job performance: Economic and social implications.
