Google Research

"I’d rather just go to bed”: Understanding Indirect Answers

Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP) (2020)


We revisit a pragmatic inference problem in dialog: Understanding indirect responses to questions. Humans can interpret I'm starving.' in response toHungry?', even without direct cue words such as yes' andno'. In dialog systems, allowing natural responses rather than closed vocabularies would be similarly beneficial. However, today's systems are only as sensitive to these pragmatic moves as their language model allows. We create and release the first large-scale English language corpus `Circa' with 34,268 (polar question, indirect answer) pairs to enable progress on this task. The data was collected via elaborate crowd-sourcing, and contains utterances with yes/no meaning, as well as uncertain, middle-ground, and conditional responses. We also present BERT-based neural models to predict such categories for a question-answer pair. We find that while transfer learning from entailment works reasonably, performance is not yet sufficient for robust dialog. Our models reach 82-88% accuracy for a 4-class distinction, and 74-85% for 6 classes.

Learn more about how we do research

We maintain a portfolio of research projects, providing individuals and teams the freedom to emphasize specific types of work