Google Research

Study on Interaction between Entropy Pruning and Kneser-Ney Smoothing

Proceedings of Interspeech (2010), pp. 2242-2245


The paper presents an in-depth analysis of a less known interaction between Kneser-Ney smoothing and entropy pruning that leads to severe degradation in language model performance under aggressive pruning regimes. Experiments in a data-rich setup such as voice search show a significant impact in WER as well: pruning Kneser-Ney and Katz models to 0.1% of their original impacts speech recognition accuracy significantly, approx. 10% relative.

Any third party with LDC membership should be able to reproduce our experiments using the scripts available at

Learn more about how we do research

We maintain a portfolio of research projects, providing individuals and teams the freedom to emphasize specific types of work