Master's Thesis

Exploration of Audio Feedback for L2 English Prosody Training

Pedro Miguel Sonso Sousa — 2021

Key information

Authors:

Pedro Miguel Sonso Sousa (Pedro Miguel Sonso Sousa)

Supervisors:

Isabel Maria Martins Trancoso (Isabel Maria Martins Trancoso); Xavier Anguera

Published in

January 28, 2021

Abstract

The increase in the number of English language learners has made using mobile apps to learn English a viable, accessible, and widely used option. This work explores two different approaches to tackle prosody training in such applications. Both methods are applied to an exercise from a language learning app, where the learner is given a sentence and a recording of a native speaker uttering this sentence. The learner then tries to read this sentence and replicate the prosodic targets from the native speaker utterance. The app returns feedback on how close the learner is to the target in terms of duration and pitch. The task will be complementing or replacing the utterance from the native speaker with an utterance in the voice of the learner. The first approach consists of manipulating the user’s speech. It will take the learner’s attempt and correct the pitch and duration markers through speech analysis with a vocoder-based system and a time alignment algorithm. The second approach uses a Voice Conversion method to convert the native speaker’s utterances to the voice of the learner. By removing the voice difference, it is expected that the learning process will be more efficient. Both approaches are implemented and preliminary results are provided. A subjective evaluation performed by a listening panel of 40 subjects is presented and a method for objective evaluation is proposed. The results reveal that the Voice Conversion approach seems the best choice for future development, given the VC algorithm is tailored for this specific task.

Publication details

Authors in the community:

Supervisors of this institution:

Fields of Science and Technology (FOS)

electrical-engineering-electronic-engineering-information-engineering - Electrical engineering, electronic engineering, information engineering

Publication language (ISO code)

eng - English

Rights type:

Embargo lifted

Date available:

December 11, 2021

Institution name

Instituto Superior Técnico