Beyond word importance: using contextual decompositions to extract interactions from LSTMs.

The driving force behind the recent success of LSTMs has been their ability to learn complex and non-linear relationships. Consequently, our inability to de- scribe these relationships has led to LSTMs being characterized as black boxes. To this end, we introduce contextual decomposition (CD), a novel algorithm for capturing the contributions of combinations of words or variables in terms of CD scores. On the task of sentiment analysis with the Yelp and SST data sets, we show that CD is able to reliably identify words and phrases of contrasting senti- ment, and how they are combined to yield the LSTM’s final prediction. Using the phrase-level labels in SST, we also demonstrate that CD is able to successfully extract positive and negative negations from an LSTM, something which has not previously been done.