An examination of errors in personality assessments in light of the replication crisis

How are different measurement errors affecting conclusions in psychological studies?

With the replication crisis, there is an increasing openness to re-examining the way measures and statistics are employed throughout the social sciences.

This project aims to examine (1) what are some common types of errors in personality assessments, and survey assessments more broadly; (2) how such errors differ between populations; and (3) how such errors can have influenced various published results in psychology and other social sciences. These questions are examined over 3-years in a collaboration between the applicant and Dr.

Jessica Kay Flake at McGill University, Canada. This is done in at least five studies.

These studies utilize large open-source datasets of personality where various aspects of measurements (data quality, measurement invariance) can be assessed across countries and other groups (e.g., adults and adolescents).

This is supplemented with simulations of errors to assess what underlying errors may reflect the variation in measurements, and surveys meant to assess how participants typically provide low-effort responses.

Finally, a literature overview that discusses specific types of errors in psychological studies is employed to illustrate their potential confound for conclusions in the field.

Understanding measurement error is crucial, because it can challenge the validity of all other steps of the research process.