While financial performance remains a critical benchmark, businesses are increasingly recognizing that DEI initiatives are not only about economic gain but also about making a broader positive impact. Embracing diversity and inclusion has evolved into a goal in itself, reflecting a commitment to broader social responsibility and a desire for ethical business practices.

At Saville Assessment, we know that psychometric assessments that are designed well and used correctly can be very effective.  However, we don’t just stop at ‘good’—our mission is to go further. We are always striving to identify new strategies to reduce bias, ensuring our assessments are fairly designed and implemented by our clients in a way that maximizes inclusivity.

Our approach to developing situational judgment assessment follows this commitment to fairness. Situational judgment tests (SJTs) gauge an individual’s understanding of appropriate responses to different job-specific scenarios. Scientific research into SJTs has uncovered interesting findings around the concept of ‘cognitive load’ – a measure of the cognitive effort and working memory required to complete a task – and its impact on test performance among different ethnic subgroups. A key finding is that SJTs with a larger cognitive loading are associated with producing larger score disparities with protected minorities disadvantaged.

While a degree of cognitive loading in an assessment may be necessary to reflect the cognitive demands of a role, any cognitive load that is extraneous can be seen as problematic, as it is unintended and arises from aspects of an assessment that are unrelated to the core task itself (i.e., judging the appropriate thing to do in a situation).

This research led us to constructing SJTs which attempt to minimize extraneous cognitive load as far as possible through reducing unnecessary complexity, in turn maximizing fairness for test-takers. We collaborated with a leading retail company to create a new SJT for their UK-wide recruitment of in-store customer service staff. The response format utilized a single-item response mechanism, where each item was presented individually. This was opposed to multiple-choice, a commonly-used format. The aim of this approach was to eliminate extraneous cognitive load, through removing the need for processing multiple stimuli simultaneously and using short-term memory to draw different comparisons. The presentation format also aimed to offer simplicity and accessibility, catering to a diverse range of users.

After 350,000 applicants completed the SJT, the results were analyzed by ethnicity. The data revealed that 52% of the total applicants were from the BAME group, and 54% of those who advanced to the next stage were also from this group. This led to a relative selection ratio of 1.08 for BAME candidates, suggesting that the test not only avoided disadvantaging minority applicants but may have also contributed to a higher proportion of these candidates progressing in the selection process. These results suggest that reducing the extraneous cognitive load of SJTs had an impressive impact on increasing assessment fairness in a practical setting.

These findings are encouraging for organizations dedicated to advancing DEI outcomes. Carefully designed SJTs present an avenue by which companies can foster a fairer selection process that supports a more diverse talent pipeline, enabling effective assessment of candidates whilst minimizing the potential for adverse impact.

Ultimately, carefully considered assessment design can have a real impact in the quest to improve DEI outcomes. Where possible, organizations should look to pair approaches like this with other steps, such as combining multiple assessment measures, reducing stereotype threats, and encouraging objective, scientific decision-making. This should enhance DEI efforts across the wider assessment process and build a more equitable and inclusive talent pipeline.