Evaluation of construct and criterion-referenced validity of a systems-thinking based near miss reporting form

The validity of methods is an ongoing issue in ergonomics. Inconsistent definitions and approaches to evaluation exacerbate this challenge. In this study, the construct and criterion-referenced validity of a new near miss reporting form was evaluated to determine the extent to which it comprehensively captures near miss incidents and is aligned with the systems thinking approach to accident causation. Interview data were used as the reference standard in the evaluation. Using signal detection theory (SDT), a high average hit rate (HR), predictive value (PV) and sensitivity index (SI) were found, with an almost perfect ranking for the index of concordance. The findings show that the reporting form has strong construct and criterion-referenced validity. It is proposed that the approach used in this study could be used by researchers and practitioners when testing the validity of incident data collection tools.

Practitioner summary: The validity of methods is a key issue in ergonomics. In this study, we test the validity of a near miss reporting form using interview data as a standard. This approach could be used by practitioners when testing the validity of other ergonomics methods.