Jump to Content
Qiujia Li

Qiujia Li

Research Areas

Authored Publications
results

Publications

Years

Research Areas

Teams

Sort By
  • Title
  • Title, descending
  • Year
  • Year, descending
    Modular Domain Adaptation for Conformer-Based Streaming ASR
    Qiujia Li
    Bo Li
    Dongseong Hwang
    Tara Sainath
    Pedro Moreno Mengibar
    Interspeech (2023)
    Preview abstract Speech data from different domains has distinct acoustic and linguistic characteristics. It is common to train a single multidomain model such as a Conformer transducer for speech recognition on a mixture of data from all domains. However, changing data in one domain or adding a new domain would require the multidomain model to be retrained. To this end, we propose a framework called modular domain adaptation (MDA) that enables a single model to process multidomain data while keeping all parameters domain-specific, i.e., each parameter is only trained by data from one domain. On a streaming Conformer transducer trained only on video caption data, experimental results show that an MDA-based model can reach similar performance as the multidomain model on other domains such as voice search and dictation by adding per-domain adapters and per-domain feed-forward networks in the Conformer encoder. View details
    Learning Word-Level Confidence for Subword End-to-End ASR
    David Qiu
    Qiujia Li
    Yanzhang (Ryan) He
    Yu Zhang
    Bo Li
    Liangliang Cao
    Rohit Prabhavalkar
    Deepti Bhatia
    Wei Li
    Ke Hu
    Tara N Sainath
    Ian Carmichael McGraw
    ICASSP (2021)
    Preview abstract We study the problem of word-level confidence estimation in subword-based end-to-end (E2E) models for automatic speech recognition (ASR). Although prior works have proposed training auxiliary confidence models for ASR systems, they do not extend naturally to systems that operate on word-pieces (WP) as their vocabulary. In particular, ground truth WP correctness labels are needed for training confidence models, but the non-unique tokenization from word to WP causes inaccurate labels to be generated. This paper proposes and studies two confidence models of increasing complexity to solve this problem. The final model uses self-attention to directly learn word-level confidence without needing subword tokenization, and exploits full context features from multiple hypotheses to improve confidence accuracy. Experiments on Voice Search and long-tail test sets show standard metrics (e.g., NCE, AUC, RMSE) improving substantially. The proposed confidence module also enables a model selection approach to combine an on-device E2E model with a hybrid model on the server to address the rare word recognition problem for the E2E model. View details
    Multi-Task Learning for E2E ASR Word and Utterance Confidence
    David Qiu
    Yanzhang (Ryan) He
    Qiujia Li
    Yu Zhang
    Liangliang Cao
    Ian Carmichael McGraw
    Interspeech (2021)
    Preview abstract Confidence scores are very useful for downstream applicationsof automatic speech recognition (ASR) systems. Recent workshave proposed using neural attention models to learn word or ut-terance confidence scores for end-to-end (E2E) ASR. By them-selves, word confidence does not model deletions, and utteranceconfidence discards much of the useful word-level training sig-nals. This paper studies the effect of adding utterance-level lossand individual deletion loss to the framework proposed in [1].Empirical results show that multi-task learning with all threeobjectives improves confidence metrics (NCE, AUC, RMSE)without the need for increasing the model size of the trans-former feature extractor. Using the utterance-level confidencefor rescoring also decreases the word error rates on Google’sVoice Search and long-tail datasets by 3-5% relative. View details
    Confidence Estimation for Attention-based Sequence-to-Sequence Models for Speech Recognition
    Qiujia Li
    David Qiu
    Yu Zhang
    Bo Li
    Yanzhang (Ryan) He
    Philip C. Woodland
    Liangliang Cao
    Trevor Deatrick Strohman
    ICASSP (2021)
    Preview abstract For various speech-related tasks, confidence scores from a speech recogniser are a useful measure to assess the quality of transcriptions. In traditional hidden Markov model-based automatic speech recognition (ASR) systems, confidence scores can be reliably obtained from word posteriors in decoding lattices. However, for an ASR system with an auto-regressive decoder such as an attention-based sequence-to-sequence model, computing word posteriors is difficult. An obvious alternative is to use the decoder softmax probability as the model confidence. To reach good recognition performance, end-to-end ASR models tend to be very large. However, large models can easily memorise training sequences, which results in overestimated confidence scores. Some regularisation techniques can directly affect softmax probabilities. In this paper, we first examine how some commonly used regularisation methods influence the confidence scores and study the overconfident behaviour of end-to-end models. Then we propose a lightweight and effective approach named confidence estimation module (CEM) on top of an existing end-to-end ASR model. Experiments on LibriSpeech show that CEM can mitigate the overconfidence problem and can produce more reliable confidence scores with and without shallow fusion of a language model. Further analysis shows that CEM generalises well to speech from a moderately mismatched domain and can potentially improve downstream tasks such as semi-supervised learning. View details
    Residual Energy-Based Models for End-to-End Speech Recognition
    Qiujia Li
    Yu Zhang
    Bo Li
    Liangliang Cao
    Philip C. Woodland
    Interspeech (2021)
    Preview abstract End-to-end models with auto-regressive decoders have shown impressive results for automatic speech recognition (ASR). These models formulate the sequence-level probability as a product of the conditional probabilities of all individual tokens given their histories. However, the performance of locally normalised models can be sub-optimal because of factors such as exposure bias. Consequently, the model distribution differs from the underlying data distribution. In this paper, the residual energy-based model (R-EBM) is proposed to complement the auto-regressive ASR model to close the gap between the two distributions. Meanwhile, R-EBMs can also be regarded as utterance-level confidence estimators, which may benefit many downstream tasks. Experiments on LibriSpeech dataset show that R-EBMs can reduce the word error rates (WERs) by 8.2%/6.7% while improving areas under precision-recall curves of confidence scores by 12.6%/28.4% on test-clean/test-other sets. Furthermore, on the state-of-the-art self-supervised learning baseline, R-EBMs also improve both recognition and confidence estimation performances significantly. View details
    Search on Google Scholar