Step-by-Step Guide for an In-Depth Medical Journal Review
Use this checklist to read a paper like a reviewer: move from a quick first pass to a structured, critical appraisal of every section.
Step 1: Preliminary Reading & Understanding Context
Abstract analysis
Does the abstract provide a clear, concise summary of the key components? Check that it briefly covers the research question, methodology, main results, and conclusion so you can grasp the study’s purpose and findings at a glance.
Context within the field
How relevant is the paper to the wider field? Does it address a genuine gap or offer a novel angle on existing work? Scan the background and references to see how well it builds on or challenges previous studies, and think about who the target audience is (clinicians, researchers, policymakers).
Step 2: In-Depth Review of Each Section
Introduction – framing the research
Research background: Does the introduction summarise key studies and current knowledge, and clearly explain why this question matters?
Clear hypothesis/objectives: Is the primary research question or hypothesis explicitly stated? Do the primary and secondary objectives match the later methods and outcomes?
Methods – assessing the methodology
Study design: Is the design appropriate for the question? (e.g. randomised controlled trial, cohort, case-control, cross-sectional). For RCTs, look for randomisation, blinding, and appropriate control groups, and consider how well the design minimises bias.
Study approach: Is analysis based on Intent-to-Treat (ITT), Per-Protocol (PP), or As-Treated? ITT preserves randomisation and is usually less biased; PP may be more explanatory but can introduce bias if many participants drop out or deviate from the protocol.
Population & sampling: Is the target population clearly defined? Are inclusion and exclusion criteria justified, and does the sample look representative? If there are multiple groups, are they well matched (e.g. age, sex, baseline risk)?
Sample-size calculation: Was a power calculation done? Underpowered studies may miss real effects and give inconclusive findings.
Detailed methods: Could another team replicate this study? Look for clear descriptions of interventions, outcome measures, timing, and tools/instruments used.
Statistical methods: Are the tests appropriate for the data type (continuous vs categorical)? Are effect sizes, confidence intervals and p-values reported?
Handling missing data: Do the authors explain how they dealt with missing data (e.g. imputation, complete-case analysis, sensitivity analyses)?
PICO checklist for methods
- Population: Who is being studied (key demographics, condition)?
- Intervention: What treatment or exposure is being tested?
- Comparison: Placebo, standard care, alternative treatment, or none?
- Outcome: What outcomes are measured (clinical, surrogate, QoL, etc.)?
Step 3: Results Evaluation
Presentation of data
Are the results organised logically, with clear tables and figures? Look for effect measures such as relative risk, odds ratios, hazard ratios, and number-needed-to-treat where appropriate.
Statistical vs clinical significance
Are findings statistically significant (often p < 0.05), and do confidence intervals exclude “no effect”? More importantly, are the differences large enough to matter clinically?
Consistency with objectives
Are all primary and secondary outcomes reported? Do the results actually answer the stated research question, or are there important gaps?
Step 4: Discussion & Interpretation
Interpretation of findings
Do the authors interpret their results in the context of existing literature? Do they compare with similar studies and explain similarities or differences?
Limitations
Are limitations openly discussed (selection bias, measurement error, confounding, small sample size, short follow-up, etc.)? Do the authors explain how these might affect the conclusions?
Clinical / practical implications
How might these findings change practice, policy, or future research? Are the conclusions cautious and proportionate to the data, or overstated?
Future directions & ethics
Do the authors suggest clear avenues for future work? Check that ethics approval and informed consent are mentioned, and that conflicts of interest and funding sources are disclosed.
Step 5: Conclusion
Does the conclusion simply summarise key findings and implications, without introducing new data? Is the tone balanced rather than over-confident?
Step 6: References & Citations
Are key foundational and recent papers cited? Is the reference list balanced, including studies that might not support the authors’ conclusions?
Step 7: Final Recommendation
Based on your review, would you recommend acceptance, minor revision, major revision, or rejection? Note specific suggestions to improve methodology, clarity of results, figures/tables, and interpretation.