Which of the following are evaluated during critical appraisal of an article?

Critical appraisal of scientific literature is a necessary skill for healthcare students. Students can be overwhelmed by the vastness of search results. Database searching is a skill in itself, but will not be covered in this blog. This blog assumes that you have found a relevant journal article to answer a clinical question. After selecting an article, you must be able to sit with the article and critically appraise it. Critical appraisal of a journal article is a literary and scientific systematic dissection in an attempt to assign merit to the conclusions of an article. Ideally, an article will be able to undergo scrutiny and retain its findings as valid.

The specific questions used to assess validity change slightly with different study designs and article types. However, in an attempt to provide a generalized checklist, no specific subtype of article has been chosen. Rather, the 20 questions below should be used as a quick reference to appraise any journal article. The first four checklist questions should be answered “Yes.” If any of the four questions are answered “no,” then you should return to your search and attempt to find an article that will meet these criteria.

Critical appraisal of…the Introduction

1. Does the article attempt to answer the same question as your clinical question?
2. Is the article recently published (within 5 years) or is it seminal (i.e. an earlier article but which has strongly influenced later developments)?
3. Is the journal peer-reviewed?
4. Do the authors present a hypothesis?

Critical appraisal of…the Methods

1. Is the study design valid for your question?
2. Are both inclusion and exclusion criteria described?
3. Is there an attempt to limit bias in the selection of participant groups?
4. Are there methodological protocols (i.e. blinding) used to limit other possible bias?
5. Do the research methods limit the influence of confounding variables?
6. Are the outcome measures valid for the health condition you are researching?

Critical appraisal of…the Results

1. Is there a table that describes the subjects’ demographics?
2. Are the baseline demographics between groups similar?
3. Are the subjects generalizable to your patient?
4. Are the statistical tests appropriate for the study design and clinical question?
5. Are the results presented within the paper?
6. Are the results statistically significant and how large is the difference between groups?
7. Is there evidence of significance fishing (i.e. changing statistical tests to ensure significance)?

Critical appraisal of…the Discussion/Conclusion

1. Do the authors attempt to contextualise non-significant data in an attempt to portray significance? (e.g. talking about findings which had a trend towards significance as if they were significant).
2. Do the authors acknowledge limitations in the article?
3. Are there any conflicts of interests noted?

This is by no means a comprehensive checklist of how to critically appraise a scientific journal article. However, by answering the previous 20 questions based on a detailed reading of an article, you can appraise most articles for their merit, and thus determine whether the results are valid. I have attempted to list the questions based on the sections most commonly present in a journal article, starting at the introduction and progressing to the conclusion. I believe some of these items are weighted heavier than others (i.e. methodological questions vs journal reputation). However, without taking this list through rigorous testing, I cannot assign a weight to them. Maybe one day, you will be able to critically appraise my future paper: How Online Checklists Influence Healthcare Students’ Ability to Critically Appraise Journal Articles.

If the answer to any of these questions is “no”, you can save yourself the trouble of reading the rest of it.

This section contains useful tools and downloads for the critical appraisal of different types of medical evidence. Example appraisal sheets are provided together with several helpful examples.

1. What is the research question? Are the objectives of the study clearly stated? Why was this research necessary?

2. Is the research original or important? Does the study have new findings? Is a treatment outcome clinically relevant?

3. Does the research question consider the following:

•     the group or population of patients
•     the intervention or therapy
•     the outcome.

4. Did the authors use the relevant type of study for the research question?

5. Did the study design minimise the risk of bias in its methodology, reporting and patient selection? Did the study use best practice design such as a randomised controlled trial or a systematic review? Blinding of patients and outcome assessors, randomization, concealment, intention-to-treat analysis, similarity of patients for known prognostic factors, and completeness of follow-up are indicators of validity.

6. Was the study designed in line with the original protocol? Is the focus of the report in keeping with the study objectives? Were changes made to the inclusion or exclusion criteria?

7. Has the study's hypothesis been tested?

8. Is the analysis of the data accurate? What level of uncertainty surrounds any results?

9. Are the conclusions based on the data and analysis? Do the authors draw conclusions that are supported by the data? Have the authors discussed other work that both supports and contradicts their findings? Have the authors identified any limitations to their study?

10. Does the study contribute to the understanding of the problem being investigated? What are the strengths and limitations of the study? Are the findings of the study useful for clinical practice? Do the risks of a treatment or diagnostic procedure outweigh the potential benefits?

What is a critical appraisal of an article?

What are the four phases of critical appraisal?

What are the three main factors of critical appraisal?

What factors does critical appraisal examine?