Display options
Share it on

J Eval Clin Pract. 2011 Aug;17(4):758-62. doi: 10.1111/j.1365-2753.2011.01693.x. Epub 2011 May 25.

The applicability of grading systems for guidelines.

Journal of evaluation in clinical practice

Adrian Baker, Jonathan Potter, Katharine Young, Ira Madan

Affiliations

  1. NHS Plus, Bristol, UK.

PMID: 21615629 DOI: 10.1111/j.1365-2753.2011.01693.x

Abstract

RATIONALE: This study focused on factors that most concern specialist societies when choosing an evidence grading system, such as methodological strengths and weaknesses, applicability and ease of use. The grading systems chosen were the Scottish Intercollegiate Guidelines Network (SIGN), the Grading of Recommendations Assessment, Development and Evaluation (GRADE) and the National Service Framework for long-term neurological conditions critical appraisal tool (NSF-LTC).

METHODOLOGY: Twelve assessors, representing typical members of society-based guideline development groups, graded papers and a recommendation using a key question as a guide. Key questions and recommendations were extracted from existing clinical guidelines representing a variety of research fields. Assessors were given 3 months to grade the papers using the grading systems and to complete a semi-structured qualitative questionnaire. The results were independently assessed for emerging themes.

RESULTS: Assessors felt all three systems had strengths and weaknesses depending on the type of evidence being graded. GRADE was seen as the most complex but rigorous system, while SIGN and NSF were seen as easier and more flexible to use, but less methodologically rigorous. In grading the evidence, 10% of assessors used GRADE incorrectly, 33% used SIGN incorrectly and 75% used NSF-LTC incorrectly. In grading the recommendations, 60% used GRADE incorrectly, 50% used SIGN incorrectly and 50% used NSF-LTC incorrectly.

IMPLICATIONS: It is recommended that specialist societies consider the type of evidence they will be evaluating and the research experience of the appraisers before selecting a grading system. Additionally, appraisers should have training in appraising and grading evidence using the system to be employed.

© 2011 Blackwell Publishing Ltd.

MeSH terms

Publication Types