Evaluating AI-Generated Versus Human-Authored Discharge Summaries in 7-to-21-Day Hospital Stays

Article Impact Level: HIGH
Data Quality: STRONG
Summary of  Npj Digital Medicine. https://doi.org/10.1038/s41746-026-03320-y 
Dr. Tyler Osborne et al.

Points

Stony Brook University researchers evaluated large language model performance in drafting discharge summaries for sixty internal medicine hospitalizations lasting between seven and twenty one days.
Paired evaluations by hospitalists and primary care physicians established that clinician reviewers preferred the artificial intelligence generated summaries in ninety five percent of evaluated patient encounters.
Comparative analyses confirmed that artificial intelligence summaries achieved superior scores for readability completeness and factuality while containing significantly fewer clinically relevant omissions than human drafts.
Reviewers assessing thirty one radiology incidental findings determined that ninety three point five percent were factually accurate and eighty seven point one percent were appropriate for discharge reporting.
Authors concluded that while physician verification remains necessary artificial intelligence integration substantially improves complex discharge documentation quality without increasing estimated clinical harm potential.

Summary

This retrospective clinician evaluation assessed the performance of a large language model (LLM) in generating hospital discharge summaries for complex, long-stay patients. Published in npj Digital Medicine by Stony Brook University researchers led by Rachel Wong and Fusheng Wang, the investigation targeted internal medicine hospitalizations lasting 7 to 21 days. Given that complex hospital stays generate extensive documentation that places a heavy burden on attending clinicians, the research sought to determine whether an LLM platform could produce discharge documentation that meets the priority needs of both inpatient hospitalists and outpatient primary care physicians (PCPs).

The study conducted a paired evaluation comparing LLM-generated and human-authored discharge summaries across 60 complex hospitalizations at Stony Brook University Hospital. Participating hospitalists and PCPs preferred the LLM-generated summaries in 95% of the evaluated encounters, assigning higher ratings for overall quality, readability, factuality, and completeness. The LLM summaries exhibited fewer clinician annotations for factual errors or clinically relevant omissions—a difference driven primarily by a reduction in omitted clinical details. In evaluating incidental radiology findings (n = 31), 93.5% were rated as factually correct and 87.1% as appropriate for inclusion in discharge communication. While errors occurred in both human and LLM documentation, physician reviewers identified no statistically significant differences in estimated potential for harm or harm likelihood.

The authors conclude that LLM-assisted documentation effectively mitigates omissions and improves readability and completeness in complex, long-stay discharge summaries without increasing potential clinical harm. Primary care physicians noted that LLM summaries simplified outpatient follow-up and patient communication. However, because both human and AI drafts contained clinical errors, human verification and oversight remain essential prerequisites before integrating LLM-generated summaries into electronic health records.

Link to the article: https://www.nature.com/articles/s41746-026-03320-y

References

Osborne, T., Mahmud, T., Zheng, X., Jampala, S., Abbasi, S., Hong, S., Kranz, K., Lee, S., Ng, P., Odekon, K., Schachter, L., Sexton, R., Spinnato, T., Tharakan, M., Wu, Z. J., Wang, F., & Wong, R. (2026). Clinician-centered evaluation of large language model-generated discharge summaries for longer hospitalizations. Npj Digital Medicine. https://doi.org/10.1038/s41746-026-03320-y