Looking at the validity and reliability of an assessment is a key way that quality tools stand out from the crowdA newly released book on the history of the Myers-Briggs Type Indicator (MBTI) has made wave for a round of press questioning the use of personality assessment for HR purposes.
Time and again, these critiques tend to fall into one of three broad themes: the tests are biased and discriminatory; they aren’t relevant for the job; or they simply fail to predict performance. However, the questions raised by all these articles seem to make one common error: It’s difficult to distinguish scientifically proven, reliable tools from those that are poor quality.
Contrary to this recent round of press, high-quality personality assessments do actually predict performance, and much better than traditional recruitment methods such as resumés, interviews and even cognitive assessments. Personality quizzes and other psychometric tests are quite trendy at the moment, so inexpensive options are everywhere. Unfortunately, you also get what you pay for.
A full 68% of European companies use assessments as part of the hiring process and evidence of validity and reliability is one of the most important factors when selecting a supplier. But in a flooded market, how can you know which assessments are actually effective?
Looking at the validity and reliability of an assessment is a key way that quality tools stand out from the crowd. Validity can tell you the predictive ability of an assessment. It measures the correlation of one thing with another: the correlation of personality with job performance. In other words, validity is a measure of accuracy. Reliability, on the other hand, is a measure of consistency. It can tell you if the assessment can properly measure the same thing time and time again.
For personality assessments, a valid tool will be able to tell you with a high degree of accuracy how well someone will perform in a particular job. Validity is measured with a coefficient between 0 and 1 (absolute value).
The closer to one, the more accurate the predictive power of the assessment. A robust assessment tool, such as the Hogan Assessment suite (HPI, HDS and MVPI) has a predictive validity of .54. Comparatively, structured interviewing of candidates has a predictive validity of only .18 – and yet, despite this, interviews remain the go-to method of selection around the world.
To put this in another context, compare the effectiveness of personality assessments to other widely accepted correlations. For example, ibuprofen is one of the most popular pain medications in the world and yet the correlation between taking it and reducing pain is .14. The effect of a coronary bypass on survival rates produces a correlation of only .08. The use of allergy medications for relieving congestion is correlated at .11.
The message is not that personality assessments are ineffective at predicting on-the-job performance. Rather, an assessment tool should be chosen with great care. Specifically, paying close attention to the following topics can help you choose a high-quality assessment tool:
- Validity and reliability – ask the vendor for information on the reliability and predictive validity of their assessments. These two things can tell you if the assessment is accurately and consistently measuring what they say it does.
- Scientific background – quality assessment tools should be heavily researched and built on a sound theoretical framework. If this information is not readily available, there’s a good chance the quality of that assessment is poor.
- Accordance with employment guidelines – many countries have employment guidelines to protect employees from discrimination. Any assessment used for recruitment purposes should demonstrate how they conform to those guidelines
- Predictive ability for job performance – often, assessments feature questions that measure identity, or self-perceptions of oneself, which can often be flawed. A better approach is to use objective measures of reputational factors that predict performance.
- Adaptability for different cultures/languages – be sure to find out if an assessment is adapted to your specific language and culture. Proper translation is important but not sufficient to account for all cultural differences.
Adrian Furnham, internationally acclaimed management expert and Professor of Psychology at University College London emphasizes:
“There are two criteria for a good assessment: evidence of test validity and quality of feedback on questionnaire. It should be useful for the employer and the employee alike: it measures clearly what you need it to measure; it is clear and straightforward for the respondent; the test has considerable evidence of reliability and validity, and the employee gets rich and useful feedback. In my experience, the three Hogan measures (HPI, HDS and MVPI) are the ones that have proved to be the most effective, because of the above reasons.”
