Skip to main content

A study of a multi-source feedback system for international medical graduates holding defined licences

Buy Article:

$51.00 plus tax (Refund Policy)



To develop and assess the feasibility and psychometric properties of multi-source feedback questionnaires to monitor international medical graduates practising in Canada under ‘defined’ licences. Method 

Four questionnaires (patient, co-worker, colleague and self) were developed and administered in 2 phases through paper-based and telephone or Internet formats. Reliability was assessed with Cronbach's alpha and generalisability coefficient analyses. Validity was established through mean ratings, ‘unable to respond’ rates and factor analyses. Results 

A total of 37 doctors participated in the 2 phases. Overall response rates were 70% for patients, 86% for co-workers, 72% for medical colleagues and 92% for self, with response rates higher for the paper-based format than the Internet and phone formats. The instruments had high internal consistency reliability, with Cronbach's alphas of 0.83 for self-assessment and > 0.90 for the other instruments. The generalisability coefficients were Ep2 = 0.71 for 25 patients on a 13-item survey, Ep2 = 0.59 for 8 co-workers on a 13-item survey, and Ep2 = 0.67 for 8 colleagues on a 21-item questionnaire. The range of mean scores was narrow (between 4 and 5) for all items and all surveys. The factor analyses identified that 2 factors accounted for 70% or more of the variance for the patient and colleague surveys and 60% of the variance for the co-worker survey. Conclusion 

These data suggest that the instruments have reasonable psychometric properties. Traditional survey methods (i.e. paper-based) yielded better results than Internet or phone methods for this group of doctors.

Keywords: *feedback; Canada; accreditation; clinical competence/*standards; education; feasibility studies; foreign medical graduates/*standards; graduate/*standards; licensure; medical; pilot projects; psychometrics; teaching/*methods

Document Type: Research Article


Affiliations: 1: Medical Council of Canada, Ottawa, Ontario, Canada 2: Department of Continuing Medical Education and Professional Development, University of Calgary, Calgary, Alberta, Canada 3: Department of Family Medicine, University of Calgary, Calgary, Alberta, Canada 4: College of Physicians and Surgeons of Saskatchewan, Saskatoon, Saskatchewan, Canada 5: College of Physicians and Surgeons of Alberta, Edmonton, Alberta, Canada 6: Children's Hospital of Eastern Ontario, Ottawa, Ontario, Canada

Publication date: 2006-04-01

  • Access Key
  • Free ContentFree content
  • Partial Free ContentPartial Free content
  • New ContentNew content
  • Open Access ContentOpen access content
  • Partial Open Access ContentPartial Open access content
  • Subscribed ContentSubscribed content
  • Partial Subscribed ContentPartial Subscribed content
  • Free Trial ContentFree trial content
Cookie Policy
Cookie Policy
Ingenta Connect website makes use of cookies so as to keep track of data that you have filled in. I am Happy with this Find out more