Daniele Pighin
Research Areas
Authored Publications
Sort By
Stepwise Extractive Summarization and Planning with Structured Transformers
Jakub Adamek
Blaž Bratanič
Ryan Thomas Mcdonald
Proceedings of The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Virtual, 4143–4159
Preview abstract
We propose encoder-centric stepwise models for extractive summarization using structured transformers -- HiBERT and Extended Transformers. We enable stepwise summarization by injecting the previously generated summary into the structured transformer as an auxiliary sub-structure. Our models are not only efficient in modeling the structure of long inputs, but they also do not rely on task-specific redundancy-aware modeling, making them a general purpose extractive content planner for different tasks. When evaluated on CNN/DailyMail extractive summarization, stepwise models achieve state-of-the-art performance in terms of Rouge without any redundancy aware modeling or sentence filtering. This also holds true for Rotowire table-to-text generation, where our models surpass previously reported metrics for content selection, planning and ordering, highlighting the strength of stepwise modeling. Amongst the two structured transformers we test, stepwise Extended Transformers provides the best performance across both datasets and sets a new standard for these challenges.
View details
Preview abstract
Accurate prediction of suitable discourse connectives (however, furthermore, etc.) is a key component of any system aimed at building coherent and fluent discourses from shorter sentences and passages. As an example, a dialog system might assemble a long and informative answer by sampling passages extracted from different documents retrieved from the Web. We formulate the task of discourse connective prediction and release a dataset of 2.9M sentence pairs separated by discourse connectives for this task. Then, we evaluate the hardness of the task for human raters, apply a recently proposed decomposable attention (DA) model to this task and observe that the automatic predictor has a higher F1 than human raters (32 vs. 30). Nevertheless, under specific conditions the raters still outperform the DA model, suggesting that there is headroom for future improvements.
View details
Redundancy Localization for the Conversationalization of Unstructured Responses
Proceedings of the SIGDIAL 2017 Conference, Association for Computational Linguistics
Preview abstract
Conversational agents offer users a naturallanguage interface to accomplish tasks, entertain themselves, or access information. Informational dialogue is particularly challenging in that the agent has to hold a conversation on an open topic, and to achieve a reasonable coverage it generally needs to digest and present unstructured information from textual sources. Making responses based on such sources sound natural and fit appropriately into the conversation context is a topic of ongoing research, one of the key issues of which is preventing the agent’s responses from sounding repetitive. Targeting this issue, we propose a new task, known as redundancy localization, which aims to pinpoint semantic overlap between text passages. To help address it systematically, we formalize the task, prepare a public dataset with fine-grained redundancy labels, and propose a model utilizing a weak training signal defined over the results of a passage-retrieval system on web texts. The proposed model demonstrates superior performance compared to a state-of-the-art entailment model and yields encouraging results when applied to a real-world dialogue.
View details
Revisiting Taxonomy Induction over Wikipedia
Preview
Amit Gupta
Francesco Piccinno
Marius Pasca
Proceedings of the 26th International Conference on Computational Linguistics (COLING-2016), Osaka, Japan, pp. 2300-2309
Idest: Learning a Distributed Representation for Event Patterns
Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL'15), pp. 1140-1149
Preview abstract
This paper describes IDEST, a new method for
learning paraphrases of event patterns. It is
based on a new neural network architecture
that only relies on the weak supervision signal
that comes from the news published on the
same day and mention the same real-world entities.
It can generalize across extractions from
different dates to produce a robust paraphrase
model for event patterns that can also capture
meaningful representations for rare patterns.
We compare it with two state-of-the-art
systems and show that it can attain comparable
quality when trained on a small dataset.
Its generalization capabilities also allow it to
leverage much more data, leading to substantial
quality improvements.
View details
Modelling Events through Memory-based, Open-IE Patterns for Abstractive Summarization
Marco Cornolti
Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL'14) (2014), pp. 892-901
Preview abstract
Abstractive text summarization of news
requires a way of representing events, such
as a collection of pattern clusters in which
every cluster represents an event (e.g.,
marriage) and every pattern in the cluster
is a way of expressing the event (e.g.,
X married Y, X and Y tied the knot). We
compare three ways of extracting event
patterns: heuristics-based, compression-based
and memory-based. While the former
has been used previously in multi-document
abstraction, the latter two have
never been used for this task. Compared
with the first two techniques, the memory-based
method allows for generating significantly
more grammatical and informative
sentences, at the cost of searching a
vast space of hundreds of millions of parse
trees of known grammatical utterances. To
this end, we introduce a data structure and
a search method that make it possible to
efficiently extrapolate from every sentence
the parse sub-trees that match against any
of the stored utterances.
View details
Preview abstract
This paper presents HEADY: a novel, ab- stractive approach for headline generation from news collections. From a web-scale corpus of English news, we mine syntactic patterns that a Noisy-OR model generalizes into event descriptions. At inference time, we query the model with the patterns observed in an unseen news collection, identify the event that better captures the gist of the collection and retrieve the most appropriate pattern to generate a headline. HEADY improves over a state-of-the- art open-domain title abstraction method, bridging half of the gap that separates it from extractive methods using human-generated titles in manual evaluations, and performs comparably to human-generated headlines as evaluated with ROUGE.
View details
