•  
  •  
 

Journal of Medical Education

Abstract

Purpose: This paper reports results from a study on constructing and validating a rubric for systematic writing assessment in the context of medical education and practice. More specifically, the rubric was designed for the assessment of reflective writing by Taiwanese healthcare students and providers. Methods: Literature related to reflective thinking and rubric development was reviewed and discussed by expert panels in order to set up a theoretical framework for the proposed Analytic Reflective Writing Scoring Rubric for Healthcare Students and Providers (ARWSR-HSP). Once the rubric was initially drafted, three experts, using the index of Item Objective Congruence (IOC), evaluated the congruence between the qualitative scoring items included in the rubric and the objectives of the ARWSRHSP. Further, the experts calculated inter-rater and intra-rater reliabilities for consistency estimates, using Spearman correlation coefficients. Weighted kappas were also calculated to confirm the inter-rater consensus estimates. Results: The IOC results revealed that among 30 qualitative criteria statements, 24 statements fell within the acceptable IOC range of 0.5 to 1.0. The remaining 6 items falling outside this range were reviewed individually in order to confirm content validity. The inter-rater reliabilities using Spearman correlation coefficients were between 0.757 and 0.946; the inter-rater reliabilities using weighed kappas were between 0.706 and 0.884. The intra-rater constancy estimates were between 0.825 and 0.930. Conclusions: The verification of content validity, IOC values, inter-rater reliabilities, and intra-rater reliabilities demonstrated that the ARWSR-HSP provides a rigorous-indeed, objective-scoring rubric for the assessment of reflective writing.

First Page

53

Last Page

72

DOI

10.6145/jme.202006_24(2).0002

Download

Share

COinS