Background: Purchasers, plans, and clinical practices involved in quality improvement initiatives are increasingly interested in measuring practice systems, particularly in relation to clinical quality and as part of pay-for-quality initiatives. The validity of self-reports of the use of practice systems was examined. Methods: In 11 medical groups in Minnesota, the Physician Practice Connections Readiness Survey, which was developed on the basis of the concepts and evidence base of the Chronic Care Model, was used to survey office practice personnel about practice systems. Participation rates by medical group ranged from 61% to 94%, with a mean of 76%, yielding surveys from 32 lead physicians and 241 other personnel. Survey results were compared with an on-site audit by trained surveyors. Results: Overall agreement with the on-site audit ranged from 40.9% to 96.7% among lead physicians and from 33.9% to 81.9% among other personnel. Mean agreement was high for quality improvement (96.7% for lead physicians and 81.9% for other personnel), moderate for clinical information systems (71.2% for lead physicians and 66.0% for others), and low for the use of care management (less than 50% for both groups). Mean positive predictive value ranged from 55.2% to 100% among lead physicians and from 49.6% to 100% among other personnel. Both the presence of systems and the accuracy of reporting varied across medical groups. Discussion: The accuracy of self-reports of practice systems varies by type of system being assessed and by type of respondent. Although self-assessment may be useful for quality improvement purposes, self-reported information on clinical practices systems should not be used for accountability purposes, including pay-for-quality efforts or public reporting unless additional documentation is required to ensure fair comparisons.
Document Type: Research Article
Publication date: July 1, 2008
More about this publication?
Published monthly, The Joint Commission Journal on Quality and Patient Safety is a peer-reviewed publication dedicated to providing health professionals with the information they need to promote the quality and safety of health care. The Joint Commission Journal on Quality and Patient Safety invites original manuscripts on the development, adaptation, and/or implementation of innovative thinking, strategies, and practices in improving quality and safety in health care. Case studies, program or project reports, reports of new methodologies or new applications of methodologies, research studies on the effectiveness of improvement interventions, and commentaries on issues and practices are all considered.
Also known as Joint Commission Journal on Quality Improvement and Joint Commission Journal on Quality and Safety