•  
  •  
 

Journal of Medical Education

Abstract

Purpose: The aim of the study was to observe the variation of rating by examiners with different personal and academic characteristics and to establish a reference data which may be used to match raters in future OSCEs in order to improve inter-rater variation. Methods: The authors reviewed the rating results during summative OSCE in 2011 for interns and Postgraduate year 1 residents (PGY1). The following eight intrinsic rater characteristics were recognized: age, gender, rater's department or subspecialty in relation to the station they rated, attending physician experience, a rater's location during examination, frequency of recent OSCE participation. These were analyzed and compared to the scores given, as well as any correlations related to them. Results: A total of ninety-seven attending physicians were included in this study. The examiners' ratings were not significantly different between different gender and the location where they rated a station. A rater, who encountered a station which was the same department or subspecialty as their own, gave marks more stringently. Older and more senior attending physicians rated significantly more leniently. The frequency and duration of recent OSCE participation in the past 2 years did not affect rating. Conclusions: Age, clinical experience, department and subspecialty of a rater were important intrinsic factors that correlated with severity of rating during an OSCE. The authors recommend that matching of the examiners with similar intrinsic characteristic to enhance inter-rater reliability.

First Page

115

Last Page

122

DOI

10.6145/jme201312

Download

Share

COinS