Journal of Medical Education
Abstract
For the past two decades remarkable changes have been implemented in medical education performance assessment (PA) reform, bringing into focus knowledge and clinical skill, but also the development of attitude, values, and interpersonal skills. Due to the emphasis on evaluating the comprehensive competencies of medical students by multiple raters, rating inconsistency inevitably occurs, with rater effects thus becoming the most serious drawback causing low inter-rater reliability and validity. This article attempts to provide a conceptual framework for and a concise update on commonly man-made errors/noises by analyzing the cause of each rater effect using a four-phase inter-rater reliability model. The author gives quantitative evidence that confirms the existence of rater effects. Four recommendations for minimizing rater effects and improving rating accuracy are offered as well.
First Page
15
Last Page
22
DOI
10.6145/jme201403
Recommended Citation
Liao, Shih-Chieh
(2014)
"Rater Effects and Corresponding Statistics for Performance Assessment,"
Journal of Medical Education: Vol. 18:
Iss.
1, Article 3.
DOI: [https://doi.org/]10.6145/jme201403
Available at:
https://jme.researchcommons.org/journal/vol18/iss1/3