Towards Better Storylines with Sentence-Level Language Models

This work proposes a sentence-level language model which predicts the next sentence in a story given the embeddings of the previous sentences. The model operates at the sentence-level and selects the next sentence within a fine set of fluent alternatives. By working with sentence embeddings instead of word embeddings, our model is able to efficiently consider a large number of alternative sentences. By considering only fluent sentences, our model is relieved from modeling fluency and can focus on longer range dependencies. Our method achieves state-of-the-art accuracy on the StoryCloze task in the unsupervised setting.