Summary
A Nature editorial argues that AI tools influencing clinical decisions should be evaluated transparently in real-world settings before deployment. It points to limited use of patient data in published evaluations and says testing should be proportionate to risk.
A Nature editorial published on 29 September 2026 argues that AI-powered medical devices used to inform clinical decisions should be tested transparently in real-world settings before deployment. The editorial points to a gap between the rapid adoption of these tools and the limited use of patient data in published evaluations, and calls for regulatory scrutiny that reflects the risk a device poses.
The US Food and Drug Administration is seeking feedback on a discussion paper about generative-AI-enabled medical devices, published in August. The editorial urges researchers to respond by the 19 October submission deadline. The paper is a request for feedback, not a final authorization rule.
Why real-world evaluation matters
AI systems are being used across clinical and administrative work, including to interpret medical images, support prescribing and help diagnose rare diseases. Some tools that record and summarize clinician–patient conversations may appear administrative, but their outputs can feed into clinical decisions. The editorial argues that this makes comprehensive evaluation before use important.
Medical products are generally assessed according to risk. As the editorial describes the current approach, manufacturers can self-certify some lower-risk products, such as bandages. Health-assessment devices such as stethoscopes may require third-party authorization, with testing in a laboratory. Products used for diagnosis that affect clinical decisions face more demanding evaluation, including testing in real-world settings and, in some cases, clinical trials similar to those used for drugs.
The editorial says AI products present a particular evaluation challenge because many are new and lack the accumulated clinical data available for established devices. It cites a systematic review published in Nature Medicine in March: of roughly 4,600 papers about AI tools for clinical medicine, 23% used real-world patient data, and only 19 studies were prospective randomized trials. These figures describe the literature covered by that review, rather than the performance of any one device.
Testing should match the device’s risk
Some evaluations compare an AI system’s judgement with that of clinicians in simulated, one-off scenarios. Such tests can offer useful information, but the editorial argues that they are not an adequate substitute for assessing devices in real clinical settings when their outputs may affect care. It also notes that studies comparing specialized medical AI systems with general-purpose models have reported conflicting results on whether specialization improves accuracy.
The editorial uses driverless cars as an analogy: their testing includes laboratory and simulated conditions as well as supervised street use, reflecting the consequences of errors. For medical AI, it does not call for full-scale randomized controlled trials for every device. Instead, it argues for applying the principles behind rigorous assessment—including transparent data and real-world testing—with the level of scrutiny guided by potential risk.
That distinction matters as regulators consider how existing medical-device frameworks should apply to generative AI. The editorial’s recommendation is a policy position, not a new clinical study; the specific requirements that regulators may adopt will depend on how they respond to feedback and develop their rules.