LIMIS: Locally Interpretable Modeling using Instance-wise Subsampling

Understanding black-box machine learning models is crucial for their widespread adoption.
Learning globally interpretable models is one approach, but achieving high performance
with them is challenging. An alternative approach is to explain individual predictions
using locally interpretable models. For locally interpretable modeling, various methods
have been proposed and indeed commonly used, but they suffer from low fidelity, i.e. their
explanations do not approximate the predictions well. In this paper, our goal is to push the
state-of-the-art in high-fidelity locally interpretable modeling. We propose a novel framework,
Locally Interpretable Modeling using Instance-wise Subsampling (LIMIS). LIMIS utilizes a
policy gradient to select a small number of instances and distills the black-box model into a
low-capacity locally interpretable model using those selected instances. Training is guided
with a reward obtained directly by measuring the fidelity of the locally interpretable models.
We show on multiple tabular datasets that LIMIS near-matches the prediction accuracy of
black-box models, significantly outperforming state-of-the-art locally interpretable models in
terms of fidelity and prediction accuracy.

• Machine Intelligence