Synthetic Data Augmentation for European Portuguese automatic speech recognition

November 10, 2025

Automatic Speech Recognition (ASR) systems have become increasingly important in our society, with applications spanning clinical documentation, transcription, voice-activated navigation, and hands-free communication. While these systems achieve exceptional performance for high-resource languages like English or Chinese, they face significant challenges for low-resource languages, such as European Portuguese, due to limited training data. The lack of diverse speech data limits the development of robust ASR systems, especially for scenarios involving different dialects, topic-specific content, or speakers of varying age groups, all of which demand high precision. To address this challenge, we propose leveraging text-to-speech (TTS) models to generate synthetic speech for data augmentation. Although reliable TTS systems for European Portuguese are lacking, zero-shot multilingual TTS systems, such as XTTS and F5-TTS, present a promising solution. In this study, we fine-tune and evaluate, these models for European Portuguese to produce speech with phonetic accuracy, prosodic naturalness, and effective style and voice cloning. Additionally, we use a large language model (LLM) to generate textual data, which is then converted into synthetic audio samples using the fine-tuned TTS models. Then, these synthetic samples can be incorporated into ASR training datasets, and their impact on recognition performance evaluated across three different areas of research. Overall, this work seeks to bridge the resource gap, contributing to the development of more effective and adaptable ASR systems for European Portuguese and establishing a foundation for future domain-specific applications.

Authors in the community:

• 

  D

  Diogo Filipe Constantino Pereira

  ist1100670

Supervisors of this institution:

• 

  A

  Alberto Abad Gareta

  ist90700

electrical-engineering-electronic-engineering-information-engineering - Electrical engineering, electronic engineering, information engineering

eng - English

Instituto Superior Técnico