Iterative Refinement for Machine Translation

Existing machine translation decoding algorithms
generate translations in a strictly
monotonic fashion and never revisit previous
decisions. As a result, earlier mistakes
cannot be corrected at a later stage. In
this paper, we present a translation scheme
that starts from an initial guess and then
makes iterative improvements that may revisit
previous decisions. We parameterize
our model as a convolutional neural network
that predicts discrete substitutions to
an existing translation based on an attention
mechanism over both the source sentence
as well as the current translation output.
By making less than one modification
per sentence, we improve the output
of a phrase-based translation system by up
to 0.4 BLEU on WMT15 German-English
translation.

• Machine Intelligence