Bruno Cartoni
Research Areas
Authored Publications
Sort By
Preview abstract
The evaluation and exchange of large lexicon databases remains a challenge in many NLP applications. Despite the existence of commonly accepted standards for the format and the features used in a lexicon, there is still a lack of precise and interoperable requirement specifications about how lexical entries of a particular language should look like, both in terms of the numbers of forms and in terms of features associated with these forms. This paper presents the notion of "lexical masks", a powerful tool used to evaluate and exchange lexicon databases in many languages.
View details
Machine Translation Evaluation beyond the Sentence Level
Jindřich Libovický
European Association for Machine Translation, Alicante, Spain (2018), pp. 179-188
Preview abstract
Automatic machine translation evaluation was crucial for the rapid development
of machine translation systems over the last two decades. So far, most
attention has been paid to the evaluation metrics that work with text on the sentence
level and so did the translation systems.
Across-sentence translation quality depends on discourse
phenomena that may not manifest at all when staying within sentence
boundaries (e.g. coreference, discourse connectives, verb tense sequence etc.).
To tackle this, we propose several document-level MT evaluation metrics:
generalizations of sentence-level metrics, language-(pair)-independent versions
of lexical cohesion scores and coreference and morphology preservation in the
target texts. We measure their agreement with human judgment on a newly
created dataset of pair-wise paragraph comparisons for four language pairs.
View details
A Database for Measuring Linguistic Information Content.
Richard Sproat
David Huynh
Linne Ha
Ravindran Rajakumar
Evelyn Wenzel-Grondie
Language Resources and Evaluation Conference, ELDA, 330 W 58th St (2014)
Preview abstract
Which languages convey the most information in a given amount of
space? This is a question often asked of linguists, especially by engineers
who often have some information theoretic measure of ``information'' in
mind, but rarely define exactly how they would measure that information. The
question is, in fact remarkably hard to answer, and many linguists consider it
unanswerable. But it is a question that seems as if it ought to have an answer.
If one had a database of close translations between a set of typologically
diverse languages, with detailed marking of morphosyntactic and morphosemantic
features, one could hope to quantify the differences between how these
different languages convey information. Since no appropriate database exists
we decided to construct one. The purpose of this paper is to present our
work on the database, along with some preliminary results. We plan to
release the dataset once complete.
View details
