Skip to main content
padlock icon - secure page this page is secure

Making the AGREE tool more user‐friendly: the feasibility of a user guide based on Boolean operators

Buy Article:

$59.00 + tax (Refund Policy)


Rationale, aims and objectives  The Appraisal of Guidelines Research and Evaluation (AGREE) instrument is a generic tool for assessing guideline quality. This feasibility study aimed to reduce the ambiguity and subjectivity associated with AGREE item scoring, and to augment the tool's capacity to differentiate between good‐ and poor‐quality guidelines.

Methods  A literature review was conducted to ascertain what AGREE instrument adjustments had been reported to date. The AGREE User Guide was then modified by:

constructing a detailed set of instructions, or dictionary, using Boolean operators, and

overlaying seven criteria to categorize guideline quality.

The feasibility of the Boolean‐based dictionary was tested by three appraisers using three randomly selected guidelines on low back pain management. The dictionary was then revised and re‐tested.

Results  Of the 52 published studies identified, 14% had modified the instrument by adding or deleting items and 35% had adopted strategies, such as using a consensus approach, to overcome inconsistencies and ensure identical item scoring among appraisers. For the feasibility test, Pearson correlation coefficients ranged from 0.27 to 0.81. Revision and re‐testing of the dictionary increased the level of agreement (range 0.41 to 0.94). Application of the revised dictionary not only decreased the variability of the domain scores, but also reduced the tool's reliability among inexperienced appraisers.

Conclusion  Appraisers found the Boolean‐based AGREE User Guide easier to use than the original, which improved their confidence in the tool. Good reliability was achieved in the feasibility test, but the reliability and validity of some of the changes will require further evaluation.
No References
No Citations
No Supplementary Data
No Article Media
No Metrics

Document Type: Research Article

Affiliations: 1: Research Associate, 2: Director, Health Technology Assessment Unit, Institute of Health Economics, Edmonton, AB, Canada

Publication date: December 1, 2009

  • Access Key
  • Free content
  • Partial Free content
  • New content
  • Open access content
  • Partial Open access content
  • Subscribed content
  • Partial Subscribed content
  • Free trial content
Cookie Policy
Cookie Policy
Ingenta Connect website makes use of cookies so as to keep track of data that you have filled in. I am Happy with this Find out more