Let’s briefly examine some basic pointers on how to perform a literature review.
If you’ve managed to get your hands on peer-reviewed articles, then you may wonder why it is necessary for you to perform your own article critique. Surely the article will be of good quality if it has made it through the peer review process?
Unfortunately this is not always the case.
Publication bias can occur when editors only accept manuscripts that have a bearing on the direction of their own research, or reject manuscripts with negative findings. Additionally, not all peer reviewers have expert knowledge of the subject matter, which can introduce bias and sometimes a conflict of interest.
Performing your own critical analysis of an article allows you to consider its value to you and to your workplace.
Critical evaluation is defined as a systematic way of considering the truthfulness of a piece of research, the results and how relevant and applicable they are.
How to Critique
It can be a little overwhelming trying to critique an article when you’re not sure where to start, however considering the article under the following headings may be of some use:
Title of Study/Research
You may be a better judge of this after reading the article, but the title should succinctly reflect the content of the work, stimulating reader’s interest.
Three to six keywords that encapsulate the main topics of the research will have been drawn from the body of the article.
This should include:
- Evidence of a literature review that is relevant and recent, critically appraising other works, not merely describing them
- Background information to the study, to orientate the reader to the problem
- Hypothesis or aims of the study
- Rationale for the study that justifies its need, i.e. to explore an un-investigated gap in the literature.
Materials and Methods
Similar to a recipe, the description of materials and methods will allow others to replicate the study elsewhere if needed. It should both contain and justify the exact specifications of selection criteria, sample size, response rate and any statistics used. This will demonstrate how the study is capable of achieving its aims. Things to consider in this section are:
- What sort of sampling technique and size was used?
- What proportion of the eligible sample participated? (e.g. “553 responded to a survey sent to 750 medical technologists”)
- Were all eligible groups sampled? (e.g. was the survey sent only in English?)
- What were the strengths and weaknesses of the study?
- Were there threats to the reliability and validity of the study and were these controlled for?
- Were there any obvious biases?
- If a trial was undertaken, was it randomised, case controlled, blinded or double-blinded?
At other times the barrier is harder, or even impossible to cross. Communication difficulties arise even when a translator is available, and non-verbal messages may be missed by the patient or even by the health professional.
Results should be statistically analysed and presented in a way that the average reader of the journal will understand. Graphs and tables should be clear and promote clarity of the text. Consider whether:
- There were any major omissions in the results, which could indicate bias
- Percentages have been used to disguise small sample sizes
- The data generated is consistent with the data collected
Negative results are just as relevant as research that produces positive results (but as mentioned previously may be omitted in publication due to editorial bias).
This should show insight into the meaning and significance of the research findings. It should not introduce any new material, but should address how the aims of the study have been met. The discussion should use previous research work and theoretical concepts as the context in which the new study can be interpreted. Any limitations of the study, including bias, should be clearly presented. You will need to evaluate whether the author has clearly interpreted the results of the study, or whether the results could be interpreted another way.
These should be clearly stated and will only be valid if the study was reliable, valid and used a representative sample size. There may also be recommendations for further research.
These should be relevant to the study, be up to date, and should provide a comprehensive list of citations within the text.
Undertaking a critique of a research article may seem challenging at first, but will help you to evaluate whether the article has relevance to your own practice and workplace. Reading a single article can act as a springboard into researching the topic more widely and aids in ensuring your nursing practice remains current and is supported by existing literature.
- Marshall, G 2005, ‘Critiquing a Research Article’, Radiography, vol. 11, pp. 55-9.
Nurses use research to answer questions about their practice, solve problems, improve the quality of patient care, generate new research questions, and shape health policy. Nurses who confront questions about practice and policy need strong, high-quality, evidence-based research. Research articles in peer-reviewed journals typically undergo a rigorous review process to ensure scholarly standards are met. Nonetheless, standards vary among reviewers and journals. This article presents a framework nurses can use to read and critique a research article.
When deciding to read an article, determine if it’s about a question you have an interest in or if it can be of use in your practice. You may want to have a research article available to read and critique as you consider the following questions.
Does the title accurately describe the article?
A good title will pique your interest but typically you will not know until you are done reading the article if the title is an accurate description. An informative title conveys the article’s key concepts, methods, and variables.
Is the abstract representative of the article?
The abstract provides a brief overview of the purpose of the study, research questions, methods, results, and conclusions. This helps you decide if it’s an article you want to read. Some people use the abstract to discuss a study and never read further. This is unwise because the abstract is just a preview of the article and may be misleading.
Does the introduction make the purpose of the article clear?
A good introduction provides the basis for the article. It includes a statement of the problem, a rationale for the study, and the research questions. When a hypothesis is being tested, it should be clearly stated and include the expected results.
Is a theoretical framework described?
When a theoretical framework is used, it should inform the study and provide a rationale. The concepts of the theoretical framework should relate to the topic and serve as a basis for interpreting the results. Some research doesn’t use a theoretical framework, such as health services research, which examines issues such as access to care, healthcare costs, and healthcare delivery. Clinical research such as comparing the effectiveness of two drugs won’t include a theoretical framework.
Is the literature review relevant to the study and comprehensive? Does it include recent research?
The literature review provides a context for the study. It establishes what is, and is not known about the research problem. Publication dates are important but there are caveats. Most literature reviews include articles published within the last 3 to 5 years. It can take more than a year for an article to be reviewed, revised, accepted, and published, causing some references to seem outdated.
Literature reviews may include older studies to demonstrate important changes in knowledge over time. In an area of study where little or no research has been conducted, there may be only a few relevant articles that are a decade or more old. In an emerging area of study there may be no published research, in which case related research should be referenced. If you are familiar with the area of research, review the references to determine if well-known and highly regarded studies are included.
Does the methods section explain how a research question was addressed?
The methods section provides enough information to allow the study to be replicated. Components of this section indicate if the design is appropriate to answer the research question(s).
- Did the researcher select the correct sample to answer the research questions and was the size sufficient to obtain valid results?
- If a data collection instrument was used, how was it created and validated?
- If any materials were used, such as written guides or equipment, were they described?
- How were data collected?
- Was reliability and validity accounted for?
- Were the procedures listed in a step-by-step manner?
Independent and dependent variables should be described and terms defined. For example, if patient falls in the hospital are considered the dependent variable, or outcome, what are the independent variables, or factors, being investigated that may influence the rate at which patient falls occur? In this example, independent variables might include nurse staffing, registered nurse composition (such as education and certification), and hospital Magnet® status.
Is the analytical approach consistent with the study questions and research design?
The analytical approach relates to the study questions and research design. A quantitative study may use descriptive statistics to summarize the data and other tests, such as chi squares, t-tests, or regression analysis, to compare or evaluate the data. A qualitative study may use such approaches as coding, content analysis, or grounded theory analysis. A reader who is unfamiliar with the analytical approach may choose to rely on the expertise of the journal’s peer reviewers who assessed whether the analytical approach was correct.
Are the results presented clearly in the text and in tables and figures?
Results should be clearly summarized in the text, tables, and figures. Tables and figures are only a partial representation of the results and critical information may be only in the text. In a quantitative study, the significance of the statistical tests is important. The presentation of qualitative results should avoid interpretation, which is reserved for the discussion.
Are the limitations presented and their implications discussed?
It is essential that the limitations of the study be presented. These are the factors that explain why the results may need to be carefully interpreted, may only be generalized to certain situations, or may provide less robust results than anticipated. Examples of limitations include a low response rate to a survey, not being able to establish causality when a cross-sectional study design was used, and having key stakeholders refuse to be interviewed.
Does the discussion explain the results in relation to the theoretical framework, research questions, and significance of the study?
The discussion serves as an opportunity to explain the results in respect to the research questions and the theoretical framework. Authors use the discussion to interpret the results and explain the meaning and significance of the study. It’s also important to distinguish the study from others that preceded it and provide recommendations for future research.
Depending on the research, it may be equally important for the investigators to present the clinical and/or practical significance of the results. Relevant policy recommendations are also important. Evaluate if the recommendations are supported by the data or seem to be more of an opinion. A succinct conclusion typically completes the article.
Once you’re done reading the article, how do you decide if the research is something you want to use?
Determine the scientific merit of the study by evaluating the level and quality of the evidence. There are many scales to use, several of which can be found in the Research Toolkit on the American Nurses Association’s website http://www.nursingworld.org/research-toolkit.aspx. Consider what you learned and decide if the study is relevant to your practice or answered your question as well as whether you can implement the findings.
A new skill
A systematic approach to reading and critiquing a research article serves as a foundation for translating evidence into practice and policy. Every nurse can acquire this skill.
Louise Kaplan is director of the nursing program at Saint Martin’s University in Lacey, Washington. At the end of this article is a checklist for evaluating an article.
Hudson-Barr D. How to read a research article. J Spec Pediatr Nurs. 2004;9(2):70-2.
King’s College D. Leonard Corgan Library. Reading a research article. http://www.lib.jmu.edu/ilworkshop08/materials/studyguide3.pdf. Accessed September 5, 2012.
Oliver D, Mahon SM. Reading a research article part I: Types of variables. Clin J Oncol Nurs. 2005;9(1):110-12.
Oliver D, Mahon SM. Reading a research article part II: Parametric and nonparametric statistics. Clin J Oncol Nurs. 2005;9(2):238-240.
Oliver D, Mahon SM. Reading a research article part III: The data collection instrument. Clin J Oncol Nurs. 2006;10(3):423-26.
Rumrill P, Fitzgerald S, Ware, M. Guidelines for evaluating research articles. Work. 2000;14(3):257-63.
Checklist for reading and critiquing a research article
1. Critiquing the research article
2. Determine the level and quality of the evidence using a scale (several can be found in ANA’s Research Toolkit http://www.nursingworld.org/Research-Toolkit/Appraising-the-Evidence ).
3. Decide if the study is applicable to your practice.