Authors: Bergus, GeorgeR; Kreiter, ClarenceD

Source: Medical Education, Volume 41, Number 7, July 2007 , pp. 661-666(6)

Publisher: Wiley-Blackwell

Abstract:

Context 

Objective structured clinical examinations (OSCEs) can be used for formative and summative evaluation. We sought to determine the generalisability of students' summary scores aggregated from formative OSCE cases distributed across 5 clerkships during Year 3 of medical school. Methods 

Five major clerkships held OSCEs with 2−4 cases each during their rotations. All cases used 15-minute student−standardised patient encounters and performance was assessed using clinical and communication skills checklists. As not all students completed every clerkship or OSCE case, the generalisability (G) study was an unbalanced student × (case : clerkship) design. After completion of the G study, a decision (D) study was undertaken and phi (φ) values for different cut-points were calculated. Results 

The data for this report were collected over 2 academic years involving 262 Year 3 students. The G study found that 9.7% of the score variance originated from the student, 3.1% from the student-clerkship interaction, and 87.2% from the student−case nested within clerkship effect. Using the variance components from the G study, the D study suggested that if students completed 3 OSCE cases in each of the 5 different clerkships, the reliability of the aggregated scores would be 0.63. The φ, calculated at a cut-point 1 standard deviation below the mean, would be approximately 0.85. Conclusions 

Aggregating case scores from low stakes OSCEs within clerkships results in a score set that allows for very reliable decisions about which students are performing poorly. Medical schools can use OSCE case scores collected over a clinical year for summative evaluation.

Keywords: clinical clerkship/*standards; programme evaluation; validity and reliability; clinical competence/*standards; judgement

Document Type: Research article

DOI: http://dx.doi.org/10.1111/j.1365-2923.2007.02786.x

Publication date: 2007-07-01

Related content

• In this: publication
• By this: publisher
• In this Subject: Medicine (General)
• By this author: Bergus, GeorgeR ;  Kreiter, ClarenceD

Website © Publishing Technology. Article copyright remains with the publisher, society or author(s) as specified within the article.

• Terms and conditions
• Privacy policy
• Information for advertisers