Adverse impact: Addressing risks of bias in job selection

One of the critical roles played by Human Resources professionals is to ensure that organisations are compliant with employment legislation, such as the case of recruitment.  Readers of this post will likely have at the least a broad understanding of legislation relating to equality at work (e.g. the Equality Act (2010) in the UK), and some may have a high level of expertise.  Beyond this, readers are likely to have values linked to promoting diversity and inclusion.  In other words, I assume readers would not deliberately discriminate against people of different ethnicities, gender identities, religious beliefs, etc.  The emphasis is on the deliberate, but this is not going to be a post about unconscious bias.  Rather, I refer to indirect discrimination, which occurs when a policy or process of some kind is applied equally, but leads to unequal outcomes for people based on protected characteristics, and is not a proportionate means for pursuing a legitimate outcome.  In other words, I want to discuss the unintended consequences of decisions that we take, known as “adverse impact.”  Adverse impact applies when, for example, we use a selection tool that discriminates against people of colour, women, individuals who are neurodiverse, and so on.  This has legal and ethical implications; from a legal perspective it is important to consider what is defensible, from an ethical perspective we should be seeking to remove unfair discrimination (based on personal characteristics rather than ability to do the job) wherever we find it.  Drawing on the research evidence helps us both legally and ethically.

It is important to consider adverse impact before making decisions.  Without a background in individual differences (a core component of psychology courses) or having read about research on recruitment and selection, it can be all too easy to make decisions that will have undesirable outcomes.  For example, intelligence (or cognitive ability as psychologists tend to call it) is one of the best predictors of job performance (Morgeson, Delaney-Klinger, & Hemingway, 2005; Salgado & Moscoso, 2019; Schmidt & Hunter, 1998).  It reflects our ability to take in information, store it, retrieve it, and use it.  People with more of it therefore learn quicker and can do more with what they learn.  It therefore seems an obvious choice for selection.  Readers are probably waiting for the “but”…  The downside is that cognitive ability tests create adverse impact based on ethnicity (Schmidt, 2002) and age (Klein, Dilchert, Ones, & Dages, 2015).  In the latter case, “fluid intelligence”, reflecting our ability to learn new things and solve new problems, declines over the adult lifespan – hence jokes about asking children to teach us how to use gadgets.  In the case of ethnicity it suffices to say that there continue to be debates as to the causes for differences, which are likely to include socioeconomic factors.

When we know about a problem, there are some points to consider.  Firstly, is this legally defensible as a proportionate way of pursuing a legitimate goal?  Continuing with the case of intelligence, it is legitimate to seek to hire the best – and no one in the diversity field is arguing otherwise.  However, we can consider whether high levels of intelligence are really essential.  Research shows that intelligence has a greater impact on performance when jobs are complex (Bertua, Anderson, & Salgado, 2005).  When hiring for a simple job we can therefore simply do without the intelligence test, because it is unnecessary, and at the same stroke do away with a major source of adverse impact.  However, we may be hiring for complex jobs in which our employees must grapple with new problems (maybe even problems we don’t know exist yet).  In this case an intelligence test may be defensible, but we should consider options for mitigating the risks.  There are a variety of strategies such as combining intelligence with other tools that have low risk of adverse impact, adjusting the weighting we give to the use of intelligence tests, or grouping performance on intelligence tests into bands based on the margin of error – within which we arguably cannot differentiate anyway (see Ployhart & Holtz, 2008 for more detail on strategies).  It is important to note that in the case of intelligence tests we are likely to face a trade-off between predicting performance and reducing adverse impact, as strategies that achieve the latter reduce our ability to predict performance.  However, we can at least make an informed decision about these trade-offs.

What I am hoping to demonstrate is that choosing a selection strategy is not a simple thing.  We need to educate ourselves about the tools that we use.  I was recently considering the use of a work sample (i.e. observing a person perform a relevant practical task) in a selection process.  They have good predictive ability (Borman & Hallam, 1991; O’Leary, Forsman, & Isaacson, 2017; Roth, Bobko, & McFarland, 2005), but research has also identified adverse impact in the use of work samples (Roth, Bobko, & McFarland, 2005).  Intuitively I had not expected this, and had I failed to review the evidence base I might not have been aware of the need to formulate a strategy for mitigating adverse impact.  Roth and colleagues observed that scholars often assumed that a work sample would show similar outcomes as actual job performance, where differences are small between people of different ethnic backgrounds.  Roth et al’s analysis supports an interpretation that some work sample tests tap cognitive ability due to the need to take in and use information, for example in-tray exercises.  They found work sample tests such as role-plays, which tap interpersonal skills, demonstrate much lower levels of adverse impact.  Thus, similar mitigation strategies might be used with work sample tests as with cognitive ability tests, starting with an evaluation of whether the test is needed.

The scenario described above is an example of the practical use of engaging with psychological literature.  However, not everyone has access to the research evidence (which is often published in academic journals) or the expertise.  If you are choosing selection tools you can take some steps to protect yourself by either gaining expertise (if you have the inclination and resources) or contracting with someone who does have the expertise.  For example, Occupational Psychologists (registered with the Health and Care Professions Council) very often have expertise in recruitment and selection techniques.

What does all this mean in practice?  We should be reviewing our practices in terms of adverse impact.  I have discussed recruitment and selection, but we can apply this lens to nearly every aspect of organisational functioning (e.g. organisational practices relating to working hours).  We also need to review decisions when we are implementing something new (e.g. making a selection process for a new job).  This will need to become a habitual part of what we do.  Racism, sexism etc. are not going to disappear quickly, or possibly ever, so we need to be sensitive to how these may occur in our structures.  New challenges may emerge, perhaps as new individual differences become known.  In other words we are going to have to create and recreate solutions, and evaluate how well they work.  We will need to scan the horizon.  For example, the use of artificial intelligence in selection poses challenges, because machine learning functions by identifying patterns without judging patterns ethically.  If our selection process discriminates against black men, and we automate the process, we now have racist automation – not what I dreamed of when reading science fiction as a child.  If we are going to do this right, we can start by recognising that it is challenging and will require effort and thought.

Dr Andrew Clements is a Lecturer in Business and Occupational Psychology at Coventry University, with interests in promoting evidence-based practice.  You can find out more about postgraduate courses teaching the application of psychology to the workplace here (for individuals with a first degree in psychology) or here (for individuals with first degrees in other subjects).