Master's Thesis
Large Language Models to Compose Discharge Summaries
— 2025
Key information
Authors:
Supervisors:
Published in
July 10, 2025
Abstract
Physicians spend approximately 45% of their time on EHR tasks, including the time-consuming process of writing discharge summaries. This burden contributes to physician burnout and reduces patient interaction time. Large Language Models (LLMs) present an opportunity to automate discharge summary generation, potentially improving clinical documentation efficiency while maintaining quality. This work explores and compares multiple open-source LLMs to automatically generate the Brief Hospital Course (BHC) and Discharge Instructions (DI) sections of discharge summaries. Three adaptation strategies were explored: In-Context Learning (ICL) using similarity-based example selection, supervised fine-tuning with QLoRA adapters, and ORPO with multiple preference selection strategies. Fine-tuning achieved the best performance, with Mistral-7B obtaining an overall score comparable to other studies, even though it used less data. One-shot learning with similarity-based example selection significantly outperformed zero-shot approaches. ORPO methods showed mixed results but demonstrated that reference-free metrics could identify synthetic examples of higher-quality than the human-written references. Error analysis revealed that reference BHC and DI sections contained unsupported medical information and scored lower than model-generated texts on completeness metrics. The findings show that LLMs can be effectively adapted for discharge summary generation. However, these also challenge the use of human-written summaries as gold standards and highlight the need for reference-free evaluation metrics in clinical NLP.
Publication details
Authors in the community:
Inês Martins Mota
ist195876
Supervisors of this institution:
Bruno Emanuel Da Graça Martins
ist24686
Fields of Science and Technology (FOS)
industrial-biotechnology - Industrial Biotechnology
Publication language (ISO code)
eng - English
Rights type:
Embargo lifted
Date available:
April 22, 2026
Institution name
Instituto Superior Técnico