mlsanityData to 14 Oct 2026

Active Learning for Speech Recognition: the Power of Gradients

arXiv 1612.03226 ↗First posted 10 Dec 2016cs.CL cs.LG stat.MLcited by 25

Not on the map. Why?

Abstract (arXiv)

In training speech recognition systems, labeling audio clips can be expensive, and not all data is equally valuable. Active learning aims to label only the most informative samples to reduce cost. For speech recognition, confidence scores and other likelihood-based active learning methods have been shown to be effective. Gradient-based active learning methods, however, are still not well-understood. This work investigates the Expected Gradient Length (EGL) approach in active learning for end-to-end speech recognition. We justify EGL from a variance reduction perspective, and observe that EGL's measure of informativeness picks novel samples uncorrelated with confidence scores. Experimentally, we show that EGL can reduce word errors by 11%, or alternatively, reduce the number of samples to label by 50%, when compared to random sampling.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). First posted on arXiv 10 Dec 2016.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2019: 1; 2020: 0; 2021: 8; 2022: 7; 2023: 2; 2024: 5; 2025: 2.

25 papers.

Sort:

By area:

  1. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  2. 2022A Survey of Active Learning for Natural Language Processingcited by 40
  3. 2021Deep Active Learning by Leveraging Training Dynamicscited by 32
  4. 2024Diversity Measurement and Subset Selection for Instruction Tuning Datasetscited by 29
  5. 2022Unifying Approaches in Active Learning and Active Sampling via Fisher Information and Information-Theoretic Quantitiescited by 28
  6. 2022Deep Active Learning with Noise Stabilitycited by 14
  7. 2021Multi-class Text Classification using BERT-based Active Learningcited by 11
  8. 2024Empowering Language Models with Active Inquiry for Deeper Understandingcited by 11
  9. 2022VL4Pose: Active Learning Through Out-Of-Distribution Detection For Pose Estimationcited by 9
  10. 2021A Neural Pre-Conditioning Active Learning Algorithm to Reduce Label Complexitycited by 7
  11. 2024Advancing Deep Active Learning & Data Subset Selection: Unifying Principles with Information-Theory Intuitionscited by 7
  12. 2022Distribution Aware Metrics for Conditional Natural Language Generationcited by 6
13 more
  1. 2023Investigating Multi-source Active Learning for Natural Language Inferencecited by 6
  2. 2025Evaluating ASR Confidence Scores for Automated Error Detection in User-Assisted Correction Interfacescited by 6
  3. 2021Deep Active Learning for Text Classification with Diverse Interpretationscited by 5
  4. 2022Smooth Sailing: Improving Active Learning for Pre-trained Language Models with Representation Smoothness Analysiscited by 4
  5. 2021Bayesian Uncertainty and Expected Gradient Length -- Regression: Two Sides Of The Same Coin?cited by 3
  6. 2023Active Learning for Natural Language Generationcited by 3
  7. 2021Active Learning to Classify Macromolecular Structures in situ for Less Supervision in Cryo-Electron Tomographycited by 2
  8. 2021Scalable Batch-Mode Deep Bayesian Active Learning via Equivalence Class Annealingcited by 2
  9. 2021ALLWAS: Active Learning on Language models in WASserstein spacecited by 1
  10. 2022Nearest Neighbor Classifier with Margin Penalty for Active Learningcited by 0
  11. 2024Towards Efficient Active Learning in NLP via Pretrained Representationscited by 0
  12. 2024Combining X-Vectors and Bayesian Batch Active Learning: Two-Stage Active Learning Pipeline for Speech Recognitioncited by 0
  13. 2025Geometric Data Valuation via Leverage Scorescited by 0