March 19, 2021

What HBO’s ’Persona’ Gets Right and (Mostly) Wrong About Personality Tests

The new documentary Persona: The Dark Side of Personality Tests is a confusing and often muddled look at the world of personality tests, but it does raise some really important issues. First, let’s look at what the documentary gets right.

The documentary focuses on a legal case first brought many years ago against Unicru, a personality test provider that once was used by Kroger and many other major retailers for customer service positions. The case centered on Kyle Behm, whose story was documented in a 2014 Wall Street Journal article. Kyle was not hired for a position at Kroger and other retailers, and he claimed it was due to his failing personality tests because of a mental health condition. (Kyle’s story later ended in tragedy, as he died by suicide in 2019).

Unicru was something of a bad actor in the field of personality testing, and the Behm lawsuit was just one in a long line of factors that led to its gradual demise. Unicru’s test featured a lot of content focused around questions that may have caused an adverse impact on people with mental illness. This would be a violation of the Americans With Disabilities Act, which prohibits administering mental health exams as a condition of employment, except in very specific circumstances. The Unicru case brought attention to the notion that certain personality tests could veer into prohibited areas, and since the creation of the ADA, the vast majority of personality test providers have generally been very careful to ensure their assessments are not measuring mental health.

The latter portion of the documentary also takes on the new AI-fueled “assessments” used by companies like HireVue. The documentary questions the use of black box algorithms that analyze job seekers’ facial expressions to draw conclusions about future performance. We have previously written about how problematic this sort of approach is, from both an ethical and a scientific perspective. The Washington Post has also raised serious questions around the use of HireVue’s technology in articles here and here. At Criteria, we believe that selection methodologies should be as transparent as possible, that candidates should have an idea of how and why they are being evaluated, and that technology providers that make hiring tools should rely on well-established peer-reviewed science. Persona is absolutely right, in my opinion, that systems like HireVue’s deserve further scrutiny.

But unfortunately, these two important points are obscured by the rest of the documentary, which yields a very misguided impression of the field of personality assessment. The documentary describes the origins of the world’s most popular personality test, the Myers-Briggs Type Indicator (MBTI), in the 1940s. It also discusses the Big Five or Five-Factor model of personality, which is the dominant model underlying most commercial preemployment assessments produced today. Persona calls the Myers-Briggs and the Big Five the “Coke and Pepsi” of the personality testing market, but drawing any equivalence between the two is terribly misleading, akin to conflating astrology and astronomy.

Check out the latest issue of Recruiter.com Magazine for more career advice and recruiting trends:

The MBTI is based on 70-year old pseudoscience, has been widely discredited from a scientific point of view, and — as the documentary itself points out — has been disavowed as a tool for making high-stakes talent decisions even by its own publishers. Its enduring popularity, despite the widespread academic consensus that its science hasn’t aged well, is testimony to the appeal of the model it envisions. Based on 16 distinct personality types, the MBTI persists because it offers an accessible and memorable, if deeply flawed, framework that provides an easy way to get to know oneself and one’s coworkers. But the MBTI clearly doesn’t meet basic psychometric standards for test reliability.

The Big Five model, on the other hand, has been empirically verified through decades of research and hundreds of peer-reviewed studies. It has emerged as the dominant modern taxonomy of personality, and it has been shown to have significant predictive utility in a wide variety of settings. As we have noted in the past, the difference between the scientific rigor behind the MBTI and the Big Five could not be starker.

Another puzzling part of the documentary is the section that features a New York City-based work readiness program called the Hope Program, which seems to be doing valuable work with underserved populations to promote work readiness. But the documentary implies that the program’s use of personality tests is a barrier to access for underserved populations. In fact, when compared to all the selection criteria that employers use to make selection decisions — including resumes, degree requirements, background checks, and interviews — professionally developed personality tests are pretty unique in having no demonstrable adverse impact against people of color. Members of underserved populations pass these tests at the same rates as other populations. The laudable focus on removing barriers to access created by structural racism is better directed at legacy hiring techniques that have been shown to lead to the disproportionate exclusion of minorities.

One line in particular stands out as misleading: The documentary claims that personality tests are normed on primarily white, male college graduates. This may have been a fair criticism when the MBTI was developed in the 1940s, but it is simply not true anymore. Reputable test publishers today go out of their way to ensure all their assessments are normed on standardization samples that reflect the demographic makeup of contemporary society.

So if you are inclined to watch Persona, you should approach it with the same spirit with which you should the MBTI: It may help you have interesting discussions at a dinner party, but don’t take all of its conclusions much more seriously than that.

Josh Millet is founder and CEO of Criteria Corp.

Read more in Assessment

Josh Millet is the founder and CEO of Los Angeles-based Criteria Corp, a preemployment assessment company with a simple goal: to help organizations make better hires. With more than 20 million assessments administered globally, the company has helped organizations make objective, data-driven hiring decisions that lead to better business outcomes with its SaaS product, HireSelect. Prior to launching Criteria, Josh cofounded an online test preparation company, Number2.com, which was acquired by XAP Corporation in 2002. After the acquisition, Josh served as president of XAP's test prep division. He has coauthored testing-related articles published in the "Journal of Educational Computing" and the "American Psychological Society Observer." Josh holds a PhD in history from Harvard University, where he was a Fulbright Scholar and a Mellon Fellow.
https://www.criteriacorp.com/