Master's Thesis
Glass-Box Quality Estimation for Neural Machine Translation
— 2021
Key information
Authors:
Supervisors:
Published in
October 1, 2021
Abstract
Quality Estimation has become increasingly relevant in the last few years for practical and confidence-aware Machine Translation applications, with recent advancements in the field of Natural Language Processing having enabled new approaches to the task. Despite the great improvements that state-of-the-art Quality Estimation systems boast, most overlook a promising source of information: the translation system under evaluation is treated as a black box, with only its input and output being regarded. In this thesis, we introduce a method which allows for the integration of information extracted from the internal mechanisms of Machine Translation models, into the training process of Quality Estimation models, which we call Glass-Box Quality Estimation. First, in order to extract this internal information, we leverage existing model uncertainty quantification methods based on Monte Carlo dropout, which recent work has shown to yield features highly relevant to estimating the quality of machine translated text. We then propose a novel model architecture based on the Predictor-Estimator framework, and an accompanying method to integrate the extracted features into the model's training procedure. Finally, we provide an empirical evaluation based on six language pairs in the context of the WMT Quality Estimation Shared Task, with encouraging results. Our analysis of the proposed model suggests various directions for future improvements.
Publication details
Authors in the community:
João Pinto Correia de Moura
ist181765
Supervisors of this institution:
André Filipe Torres Martins
ist46911
João Miguel Da Costa Sousa
ist12897
Fields of Science and Technology (FOS)
mechanical-engineering - Mechanical engineering
Publication language (ISO code)
por - Portuguese
Rights type:
Embargo lifted
Date available:
August 19, 2022
Institution name
Instituto Superior Técnico