Khabre branded placeholder image

health

KampoBench Tests LLMs on Japanese Medicine Consultations

A medRxiv preprint introduces KampoBench, a physician-revised benchmark testing 10 large language models on 74 Japanese Kampo consultation scenarios. The results point to a gap between naming relevant information and applying it to a specific case.

Khabre branded placeholder image

health

Simulated patient tests reveal gaps in AI intake chatbot conversations

A medRxiv preprint reports a remote, voice-based method for testing patient-facing AI intake chatbots with standardized-patient encounters. In a pilot, the chatbot covered fewer critical history elements in harder cases, while summary ratings remained near ceiling.

Khabre branded placeholder image

health

Clinical AI benchmark finds gains from more matched evidence

A preprint reports that four large language models more often matched clinical questions to supporting evidence when given access to a larger corpus of precision evidence-based findings. The evaluation used 209 synthetic questions and measured evidence alignment and faithful reporting.

Khabre branded placeholder image

health

Study finds word error rate can miss high-risk AI scribe errors

A medRxiv preprint evaluated an ambient AI scribe across a multilingual, controlled corpus and found that word error rate did not reliably identify rare, clinically consequential transcription errors. The authors argue that frequency-based accuracy measures should be complemented by context-aware clinical-risk assessment.