David Vilar
David Vilar is a Research Scientist working for Google Translate since 2020. He has been working on machine translation research since 2003, and was co-author of the open source Jane (developed by RWTH Aachen) and Sockeye (developed by Amazon) systems.
Research Areas
Authored Publications
Sort By
There's no Data Like Better Data: Using QE Metrics for MT Data Filtering
Jan-Thorsten Peter
Mara Finkelstein
Jurik Juraska
Proceedings of the Eighth Conference on Machine Translation, Association for Computational Linguistics, Singapore (2023), pp. 561-577
Preview abstract
Quality Estimation (QE), the evaluation of machine translation output without the need of explicit references, has seen big improvements in the last years with the use of neural metrics. In this paper we analyze the viability of using QE metrics for filtering out bad quality sentence pairs in the training data of neural machine translation systems (NMT). While most corpus filtering methods are focused on detecting noisy examples in collections of texts, usually huge amounts of web crawled data, QE models are trained to discriminate more fine-grained quality differences. We show that by selecting the highest quality sentence pairs in the training data, we can improve translation quality while reducing the training size by half. We also provide a detailed analysis of the filtering results, which highlights the differences between both approaches.
View details
Prompting PaLM for Translation: Assessing Strategies and Performance
Jiaming Luo
Viresh Ratnakar
George Foster
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association for Computational Linguistics, Toronto, Canada (2023), 15406–15427
Preview abstract
Large language models (LLMs) that have been trained on multilingual but not parallel text exhibit a remarkable ability to translate between languages. We probe this ability in an in-depth study of the pathways language model (PaLM), which has demonstrated the strongest machine translation (MT) performance among similarly-trained LLMs to date. We investigate various strategies for choosing translation examples for few-shot prompting, concluding that example quality is the most important factor. Using optimized prompts, we revisit previous assessments of PaLM’s MT capabilities with more recent test sets, modern MT metrics, and human evaluation, and find that its performance, while impressive, still lags that of state-of-the-art supervised systems. We conclude by providing an analysis of PaLM’s MT output which reveals some interesting properties and prospects for future work.
View details
A Natural Diet: Towards Improving Naturalness of Machine Translation Output
David Grangier
George Foster
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Online (2022)
Preview abstract
Machine translation (MT) evaluation often focuses on accuracy and fluency, without paying much attention to translation style. This means that, even when considered accurate and fluent, MT output can still sound less natural than high quality human translations or text originally written in the target language. Machine translation output notably exhibits lower lexical diversity, and employs constructs that mirror those in the source sentence. In this work we propose a method for training MT systems to achieve a more natural style, i.e. mirroring the style of text originally written in the target language. Our method tags parallel training data according to the naturalness of the target side by contrasting language models trained on natural and translated data. Tagging data allows us to put greater emphasis on target sentences originally written in the target language. Automatic metrics show that the resulting models achieve lexical richness on par with human translations, mimicking a style much closer to sentences originally written in the target language. Furthermore, we find that their output is preferred by human experts when compared to the baseline translations.
View details
Preview abstract
We address efficient calculation of influence functions (Koh & Liang 2017) for tracking predictions back to the training data. We propose and analyze a new approach to speeding up the inverse Hessian calculation based on Arnoldi iteration (Arnoldi 1951). With this improvement, we achieve, to the best of our knowledge, the first successful implementation of influence functions that scales to full-size (language and vision) Transformer models with several hundreds of millions of parameters. We evaluate our approach in image classification and sequence-to-sequence tasks with tens to a hundred of millions of training examples. Our implementation will be publicly available at https://github.com/google-research/jax-influence.
View details
Controlling Machine Translation for Multiple Aspects with Additive Interventions
Andrea Schioppa
Artem Sokolov
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Online and Punta Cana, Dominican Republic, pp. 6676-6696
Preview abstract
Fine-grained control of machine translation (MT) outputs along multiple aspects is critical for many modern MT applications and is a requirement for gaining users' trust. A standard approach for exerting control in MT is to prepend the input with a special tag to signal the desired output aspect. Despite its simplicity, aspect tagging has several drawbacks: continuous values must be binned into discrete categories, which is unnatural for certain applications; interference between multiple tags is poorly understood and needs fine-tuning. We address these problems by introducing vector-valued interventions which allow for fine-grained control over multiple aspects simultaneously via a weighted linear combinations of the corresponding vectors. For some aspects, our approach even allows for fine-tuning a model trained without annotations to support such interventions. In experiments with three aspects (length, politeness and monotonicity) and two language pairs (English to German and Japanese) our models achieve better control over a wider range of tasks compared to tagging, and translation quality does not degrade when no control is requested. Finally, we demonstrate how to enable control in an already trained model after a relatively cheap fine-tuning stage.
View details
Preview abstract
Training data for machine translation (MT) is often sourced from a multitude of large corpora that are multi-faceted in nature, e.g. containing contents from multiple domains or different levels of quality or complexity. Naturally, these facets do not occur with equal frequency, nor are they equally important for the test scenario at hand.
In this work, we propose to optimize this balance jointly with MT model parameters to relieve system developers from manual schedule design.
A multi-armed bandit is trained to dynamically choose between facets in a way that is most beneficial for the MT system.
We evaluate it on three different multi-facet applications: balancing translationese and natural training data, or data from multiple domains or multiple language pairs. We find that bandit learning leads to competitive MT systems across tasks, and our analysis provides insights into its learned strategies and the underlying data sets.
View details
