Summary
A medRxiv preprint comparing APOLLAI with emergency medical services raters on recordings from 295 patients with suspected stroke reports higher sensitivity for the AI system across five neurological tasks. The authors say prospective validation in real-world EMS conditions is needed.
A medRxiv preprint reports that APOLLAI, a multimodal artificial intelligence system, was more sensitive than emergency medical services (EMS) personnel at identifying five stroke-related neurological deficits in patient recordings. In a prospective study of 295 people with suspected stroke, the system’s sensitivity for the overall assessment was 78.5%, compared with 59.4% for EMS raters; specificity was similar. The study evaluated recognition of selected deficits against expert consensus labels, rather than whether AI use improves stroke diagnosis or patient outcomes.
How the comparison was conducted
The recordings were obtained at Nîmes University Hospital between August 2023 and July 2025. Participants had a mean age of 70.3 ± 13.3 years. Seven certified stroke neurologists independently assessed five items from the National Institutes of Health Stroke Scale: best gaze, facial palsy, arm motor function, aphasia and dysarthria. Their assessments were combined into consensus labels using the Dawid-Skene algorithm. EMS personnel also rated the recordings, with a mean of 15 raters per patient.
APOLLAI combines computer vision for video, deep learning for audio and a support vector machine (SVM) that fuses those inputs into a final classification. The researchers compared the system and EMS ratings with the neurologists’ consensus using sensitivity, specificity and the area under the receiver operating characteristic curve (AUC). Sensitivity measures how often an assessment identifies a deficit when the reference labels indicate one; specificity measures how often it correctly identifies its absence.
Results and their scope
The study reports higher APOLLAI sensitivity than EMS for each of the five neurological tasks and for the global assessment. For that overall assessment, specificity was 87.0% for APOLLAI and 85.9% for EMS. APOLLAI’s global AUC was 0.882; task-level AUCs ranged from 0.839 for facial palsy to 0.966 for best gaze. AUC summarises how well a system distinguishes positive from negative reference labels across possible decision thresholds.
Agreement among the seven neurologists was moderate overall, with a mean κ of 0.541 and a range of 0.367–0.732. That variability underscores the difficulty of assessing neurological signs consistently, including among certified clinicians.
The findings make APOLLAI a candidate adjunct for standardising stroke-deficit recognition, but the comparison was based on recordings collected at a university hospital. The authors call for prospective validation under real-world EMS conditions before its performance in operational prehospital care can be assessed. The report is a medRxiv preprint.