Summary
A BMJ research methods paper presents INSPECT-SR, a 21-check tool for assessing the trustworthiness of randomised controlled trials in systematic reviews. It guides reviewers to judge concerns across four domains, but is not a fraud-detection or risk-of-bias tool.
A BMJ research methods paper presents INSPECT-SR, a tool that guides systematic reviewers through up to 21 checks to assess the trustworthiness of randomised controlled trials (RCTs). Its purpose is to help reviewers identify studies with serious concerns before those trials are included in evidence syntheses, where problematic data could affect conclusions about treatments.
INSPECT-SR addresses a different question from a risk-of-bias assessment. Risk-of-bias tools examine whether a trial’s design or conduct could have distorted its results; INSPECT-SR checks whether there are serious questions about the trustworthiness of the reported methods or findings. The authors emphasise that concerns identified by the tool are not, by themselves, accusations of research misconduct.
Four domains, with reviewer judgement
The checks are grouped into four domains: post-publication notices; trial conduct, governance and transparency; text and figures; and results reported across the study. A “yes” response to a check signals a potential problem. Reviewers then judge each domain as having “no concerns,” “some concerns” or “serious concerns,” and use those judgements to reach an overall assessment.
The tool does not turn check responses into an automatic score. Reviewers use the accompanying guidance to interpret each check, explain their reasoning and decide how much the findings affect confidence in a trial. The overall assessment should usually be at least as severe as the most concerning domain judgement. Multiple domains rated “some concerns” may also justify a study-level judgement of “serious concerns.”
INSPECT-SR can be used across healthcare areas and does not require access to individual participant data. It does not assess risk of bias, generalisability or conflicts of interest. The authors also caution against treating it as a diagnostic test for fraud: serious concerns may arise from misconduct or from critical errors in an otherwise honest trial.
How the tool was developed and tested
The development combined a review of potential checks, application of checks to trials in Cochrane reviews, a two-round Delphi survey, consensus meetings and user testing. In the Delphi survey, 148 participants completed the first round and 134 the second. A check met the prespecified consensus threshold if at least 80% of participants rated its usefulness 7 or higher on a 1-to-9 scale. Twenty-nine checks met that threshold and went forward to consensus meetings; the final tool contains 21.
For user testing, 40 people provided feedback after assessing one or more trials. The median reported assessment time was 45 minutes per trial, with an interquartile range of 27 to 74 minutes. Of these testers, 28 said they would use the tool in future work, and 37 considered the judgements reached reasonable. The paper notes that testers were a self-selected group, so those responses may overstate how acceptable the tool would be to users more broadly.
How systematic reviewers are advised to use it
The authors recommend applying INSPECT-SR to eligible trials before risk-of-bias assessment and data extraction. They advise that two reviewers assess each trial independently and then reach agreement, with at least one reviewer bringing subject expertise. Trials judged to have “serious concerns” should be excluded from the systematic review; trials with “some concerns” should be examined in sensitivity analyses to see how their inclusion affects results.
The tool and its detailed guidance provide a structured way to document why a trial raises concerns. Its application still depends on reviewers’ interpretation, and the authors identify evaluation of agreement between reviewer pairs as an important part of assessing how the tool works in practice.