Master's Thesis

Large Language Models to Compose Discharge Summaries

Inês Martins Mota — 2025

Key information

Authors:

Inês Martins Mota (Inês Martins Mota)

Supervisors:

Gonçalo Migueis de Matos Afonso Correia; Bruno Emanuel Da Graça Martins (Bruno Emanuel Da Graça Martins)

Published in

July 10, 2025

Abstract

Physicians spend approximately 45% of their time on EHR tasks, including the time-consuming process of writing discharge summaries. This burden contributes to physician burnout and reduces patient interaction time. Large Language Models (LLMs) present an opportunity to automate discharge summary generation, potentially improving clinical documentation efficiency while maintaining quality. This work explores and compares multiple open-source LLMs to automatically generate the Brief Hospital Course (BHC) and Discharge Instructions (DI) sections of discharge summaries. Three adaptation strategies were explored: In-Context Learning (ICL) using similarity-based example selection, supervised fine-tuning with QLoRA adapters, and ORPO with multiple preference selection strategies. Fine-tuning achieved the best performance, with Mistral-7B obtaining an overall score comparable to other studies, even though it used less data. One-shot learning with similarity-based example selection significantly outperformed zero-shot approaches. ORPO methods showed mixed results but demonstrated that reference-free metrics could identify synthetic examples of higher-quality than the human-written references. Error analysis revealed that reference BHC and DI sections contained unsupported medical information and scored lower than model-generated texts on completeness metrics. The findings show that LLMs can be effectively adapted for discharge summary generation. However, these also challenge the use of human-written summaries as gold standards and highlight the need for reference-free evaluation metrics in clinical NLP.

Publication details

Authors in the community:

Supervisors of this institution:

Fields of Science and Technology (FOS)

industrial-biotechnology - Industrial Biotechnology

Publication language (ISO code)

eng - English

Rights type:

Embargo lifted

Date available:

April 22, 2026

Institution name

Instituto Superior Técnico