Summary
Researchers introduced EVLF-FM, a medical vision-language model that diagnoses images, answers questions and identifies relevant image regions. Its evaluations covered more than 1.3 million development and test samples, with external validation on 13,443 samples.
Researchers have introduced EVLF-FM, a medical vision-language model designed to diagnose images, answer clinical questions and point to image regions relevant to its output. The study, published in Nature Medicine on 29 October 2026, reports development and testing across more than 1.3 million samples from 18 global datasets, covering 10 imaging modalities and six specialties.
The model’s evaluations included external validation on 13,443 independent samples from 17 additional datasets. The authors describe EVLF-FM as a research model for clinical decision support; they say further validation and prospective clinical studies are needed before routine deployment.
How EVLF-FM connects diagnoses to image regions
Many medical image models are built for a particular imaging type or task. EVLF-FM is intended to handle a wider range, including CT, MRI, X-rays, ultrasound, PET, histopathology, dermatoscopy, fundus photographs, optical coherence tomography and endoscopic images. The six specialties covered were dermatology, hepatology, ophthalmology, pathology, pulmonology and radiology.
Its architecture combines two kinds of image information. A disease-level encoder extracts broader features that can support diagnosis, while a pixel-level encoder retains spatial detail. The model supplies both to a language model, which can produce a diagnosis or answer, generate a text explanation, and localise a finding in the image. These capabilities are tested as image diagnosis, visual question answering and medical visual grounding.
Training used supervised fine-tuning followed by visual reinforcement fine-tuning. The latter uses task-specific rewards, including rewards based on whether a predicted region overlaps with an expert-marked region. The researchers also describe a sentence-level method for aligning generated explanations with reference clinical reasoning.
In internal diagnostic validation, EVLF-FM achieved a reported mean accuracy of 0.895 across the evaluated tasks, the highest average among the models compared. For visual grounding across nine imaging modalities, its average accuracy at an intersection-over-union threshold of 0.5 was 0.842. Intersection over union measures how much a predicted region overlaps with the reference region; a higher score indicates closer localisation.
What the evaluations show—and what they do not yet answer
The paper reports competitive zero-shot and few-shot results on external datasets. In few-shot testing, the model receives a small number of labelled examples for a task; performance generally improved as examples were added, although the best result did not always occur with the largest example set.
The researchers also asked 12 physicians—four junior doctors and eight specialists—to assess generated explanations for sampled cases from five imaging modalities. Using the S.C.O.R.E. framework, which rates safety, consensus and context, objectivity, reproducibility and explainability, the outputs received an overall mean score of 4.25 out of 5. This is an assessment of model-generated explanations by clinicians, not a measure of patient outcomes. The explanations are post hoc: they accompany the model’s predictions rather than providing a direct record of its internal computation.
The authors identify important steps before clinical use. EVLF-FM currently handles static two-dimensional image inputs and does not explicitly process volumetric scans. They also say its accuracy is not yet at the level typically needed for direct clinical deployment, and call for validation in more diverse healthcare settings and prospective studies of safety, effectiveness and clinical utility. The reported results therefore describe benchmark performance and clinician assessment, not demonstrated benefit in routine patient care.