Methodological and Reporting Quality of Systematic Reviews Published in the Highest Ranking Journals in the Field of Pain

BACKGROUND:

Systematic reviews (SRs) are important for making clinical recommendations and guidelines. We analyzed methodological and reporting quality of pain-related SRs published in the top-ranking anesthesiology journals.

METHODS:

This was a cross-sectional meta-epidemiological study. SRs published from 2005 to 2015 in the first quartile journals within the Journal Citation Reports category Anesthesiology were analyzed based on the Journal Citation Reports impact factor for year 2014. Each SR was assessed by 2 independent authors using Assessment of Multiple Systematic Reviews (AMSTAR) and Preferred Reporting Items of Systematic reviews and Meta-Analyses (PRISMA) tools. Total score (median and interquartile range, IQR) on checklists, temporal trends in total score, correlation in total scores between the 2 checklists, and variability of those results between journals were analyzed.

RESULTS:

A total of 446 SRs were included. Median total score of AMSTAR was 6/11 (IQR: 4–7) and of PRISMA 18.5/27 (IQR: 15–22). High compliance (reported in over 90% SRs) was found in only 1 of 11 AMSTAR and 5 of 27 PRISMA items. Low compliance was found for the majority of AMSTAR and PRISMA individual items. Linear regression indicated that there was no improvement in the methodological and reporting quality of SRs before and after the publication of the 2 checklists (AMSTAR: F(1,8) = 0.22; P = .65, PRISMA: F(1,7) = 0.22; P = .47). Total scores of AMSTAR and PRISMA had positive association (R = 0.71; P < .0001).

CONCLUSIONS:

Endorsement of PRISMA in instructions for authors was not a guarantee of compliance. Methodological and reporting quality of pain-related SRs should be improved using relevant checklists. This can be remedied by a joint effort of authors, editors, and peer reviewers.