Pat Rondon
Research Areas
Authored Publications
Sort By
Improving Automatic Speech Recognition with Neural Embeddings
Christopher Li
2021 IEEE International Conference on Acoustics, Speech, and Signal Processing, 111 8th Ave
New York, NY 10011 (2021)
Preview abstract
A common challenge in automatic speech recognition (ASR) systems is successfully decoding utterances containing long tail entities. Examples of entities include unique contact names and local restaurant names that may be out of vocabulary, and therefore absent from the training set. As a result, during decoding, such entities are assigned low likelihoods by the model and are unlikely to be recognized. In this paper, we apply retrieval in an embedding space to recover such entities. In the aforementioned embedding space, embedding representations of phonetically similar entities are designed to be close to one another in cosine distance. We describe the neural networks and the infrastructure to produce such embeddings. We also demonstrate that using neural embeddings improves ASR quality by achieving an over 50% reduction in word error rate (WER) on evaluation sets for popular media queries.
View details
An Efficient Streaming Non-Recurrent On-Device End-to-End Model with Improvements to Rare-Word Modeling
Rami Botros
Ruoming Pang
David Johannes Rybach
James Qin
Quoc-Nam Le-The
Anmol Gulati
Cal Peyser
Chung-Cheng Chiu
Emmanuel Guzman
Jiahui Yu
Qiao Liang
Wei Li
Yu Zhang
Interspeech (2021) (to appear)
Preview abstract
On-device end-to-end (E2E) models have shown improvementsover a conventional model on Search test sets in both quality, as measured by Word Error Rate (WER), and latency, measured by the time the result is finalized after the user stops speaking. However, the E2E model is trained on a small fraction of audio-text pairs compared to the 100 billion text utterances that a conventional language model (LM) is trained with. Thus E2E models perform poorly on rare words and phrases. In this paper, building upon the two-pass streaming Cascaded Encoder E2E model, we explore using a Hybrid Autoregressive Transducer (HAT) factorization to better integrate an on-device neural LM trained on text-only data. Furthermore, to further improve decoder latency we introduce a non-recurrent embedding decoder, in place of the typical LSTM decoder, into the Cascaded Encoder model. Overall, we present a streaming on-device model that incorporates an external neural LM and outperforms the conventional model in both search and rare-word quality, as well as latency, and is 318X smaller.
View details
Preview abstract
End-to-end (E2E) mixed case automatic speech recognition systems
(ASR) that directly predict words in the written domain are attractive
due to being simple to build, not requiring explicit capitalization
models, allowing streaming capitalization without additional effort
beyond that required for streaming ASR, and their small size.
However, the fact that these systems produce various versions of the same
word with different capitalizations, and even different word
segmentations for different case variants when wordpieces (WP) are predicted,
leads to multiple problems with contextual ASR. In particular,
the size and time to build contextual models grows considerably
with the number of variants per word. In this paper, we propose
separating orthographic recognition from capitalization, so that the
ASR system first predicts a word, then predicts its capitalization in
the form of a capitalization mask. We show that the use of capitalization
masks achieves the same low error rate as traditional mixed
case ASR, while reducing the size and compilation time of contextual models.
Furthermore, we observe significant improvements in capitalization quality.
View details
Entropy Based Pruning of Backoff MaxEnt Language Models with Contextual Features
Tongzhou Chen
Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Proceesing, Calgary, Canada (2018)
Preview abstract
In this paper, we present a pruning technique for maximum entropy (MaxEnt)
language models. It is based on computing the exact entropy loss when removing
each feature from the model, and it explicitly supports backoff features by
replacing each removed feature with its backoff.
The algorithm computes the loss on the training data, so it is not restricted to
models with n-gram like features, allowing models with any feature, including
long range skips, triggers, and contextual features such as device location.
Results on the 1-billion word corpus show large perplexity improvements
relative for frequency pruned models of comparable size.
Automatic speech recognition (ASR) experiments show up to 0.2\% absolute WER
improvements in a large-scale cloud based mobile ASR system for Italian.
View details
