J Eval Clin Pract. 2011 Aug;17(4):758-62. doi: 10.1111/j.1365-2753.2011.01693.x. Epub 2011 May 25.
The applicability of grading systems for guidelines.
Journal of evaluation in clinical practice
Adrian Baker, Jonathan Potter, Katharine Young, Ira Madan
PMID: 21615629
DOI: 10.1111/j.1365-2753.2011.01693.x
Abstract
RATIONALE: This study focused on factors that most concern specialist societies when choosing an evidence grading system, such as methodological strengths and weaknesses, applicability and ease of use. The grading systems chosen were the Scottish Intercollegiate Guidelines Network (SIGN), the Grading of Recommendations Assessment, Development and Evaluation (GRADE) and the National Service Framework for long-term neurological conditions critical appraisal tool (NSF-LTC).
METHODOLOGY: Twelve assessors, representing typical members of society-based guideline development groups, graded papers and a recommendation using a key question as a guide. Key questions and recommendations were extracted from existing clinical guidelines representing a variety of research fields. Assessors were given 3 months to grade the papers using the grading systems and to complete a semi-structured qualitative questionnaire. The results were independently assessed for emerging themes.
RESULTS: Assessors felt all three systems had strengths and weaknesses depending on the type of evidence being graded. GRADE was seen as the most complex but rigorous system, while SIGN and NSF were seen as easier and more flexible to use, but less methodologically rigorous. In grading the evidence, 10% of assessors used GRADE incorrectly, 33% used SIGN incorrectly and 75% used NSF-LTC incorrectly. In grading the recommendations, 60% used GRADE incorrectly, 50% used SIGN incorrectly and 50% used NSF-LTC incorrectly.
IMPLICATIONS: It is recommended that specialist societies consider the type of evidence they will be evaluating and the research experience of the appraisers before selecting a grading system. Additionally, appraisers should have training in appraising and grading evidence using the system to be employed.
© 2011 Blackwell Publishing Ltd.
MeSH terms
Publication Types