mlsanityData to 14 Oct 2026

Tree-constrained Pointer Generator with Graph Neural Network Encodings for Contextual Speech Recognition

arXiv 2207.00857 ↗First posted 2 Jul 2022cs.SD cs.CL eess.AScited by 17

Not on the map. Why?

Abstract (arXiv)

Incorporating biasing words obtained as contextual knowledge is critical for many automatic speech recognition (ASR) applications. This paper proposes the use of graph neural network (GNN) encodings in a tree-constrained pointer generator (TCPGen) component for end-to-end contextual ASR. By encoding the biasing words in the prefix-tree with a tree-based GNN, lookahead for future wordpieces in end-to-end ASR decoding is achieved at each tree node by incorporating information about all wordpieces on the tree branches rooted from it, which allows a more accurate prediction of the generation probability of the biasing words. Systems were evaluated on the Librispeech corpus using simulated biasing tasks, and on the AMI corpus by proposing a novel visual-grounded contextual ASR pipeline that extracts biasing words from slides alongside each meeting. Results showed that TCPGen with GNN encodings achieved about a further 15% relative WER reduction on the biasing words compared to the original TCPGen, with a negligible increase in the computation cost for decoding.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 27 of its 42 references that are in the map. First posted on arXiv 2 Jul 2022.

Builds on

27 of 42 references.

References · 27

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  3. 2017Graph Attention Networkscited by 8,923
  4. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  5. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  6. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,292
  7. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 669
  8. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 558
  9. 2015Listen, Attend and Spellcited by 543
  10. 2018Deep context: end-to-end contextual speech recognitioncited by 137
  11. 2021Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusioncited by 101
  12. 2020Contextual RNN-T For Open Domain ASRcited by 85
15 more
  1. 2020A Density Ratio Approach to Language Model Fusion in End-To-End Automatic Speech Recognitioncited by 73
  2. 2020Deep Shallow Fusion for RNN-T Personalizationcited by 64
  3. 2020Class LM and word mapping for contextual biasing in End-to-End ASRcited by 42
  4. 2022Improving End-to-End Contextual Speech Recognition with Fine-Grained Contextual Knowledge Selectioncited by 39
  5. 2021Tree-constrained Pointer Generator for End-to-end Contextual Speech Recognitioncited by 32
  6. 2020Joint Contextual Modeling for ASR Correction and Language Understandingcited by 30
  7. 2021Instant One-Shot Word-Learning for Context-Specific Neural Sequence-to-Sequence Speech Recognitioncited by 30
  8. 2018Contextual Speech Recognition with Difficult Negative Training Examplescited by 28
  9. 2018End-to-end contextual speech recognition using class language models and a token passing decodercited by 26
  10. 2020Contextualizing ASR Lattice Rescoring with Hybrid Pointer Network Language Modelcited by 11
  11. 2018Contextual Language Model Adaptation for Conversational Agentscited by 10
  12. 2019Hierarchical Character Embeddings: Learning Phonological and Semantic Representations in Languages of Logographic Origin using Recursive Neural Networkscited by 8
  13. 2020Fast and Robust Unsupervised Contextual Biasing for Speech Recognitioncited by 7
  14. 2022Neural-FST Class Language Model for End-to-End Speech Recognitioncited by 6
  15. 2018Recursive Neural Network Based Preordering for English-to-Japanese Machine Translationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 5; 2024: 3; 2025: 6; 2026: 2.

17 papers.

Sort:

By area:

  1. 2023Can Contextual Biasing Remain Effective with Whisper and GPT-2?cited by 21
  2. 2024CTC-Assisted LLM-Based Contextual ASRcited by 19
  3. 2026SLAM-LLM: A Modular, Open-Source Multimodal Large Language Model Framework and Best Practice for Speech, Language, Audio and Music Processingcited by 8
  4. 2023ed-cec: improving rare word recognition using asr postprocessing based on error detection and context-aware error correctioncited by 7
  5. 2022End-to-end Spoken Language Understanding with Tree-constrained Pointer Generatorcited by 6
  6. 2023Graph Neural Networks for Contextual ASR with the Tree-Constrained Pointer Generatorcited by 6
  7. 2023Towards Effective and Compact Contextual Representation for Conformer Transducer Speech Recognition Systemscited by 6
  8. 2025Do Slides Help? Multi-modal Context for Automatic Transcription of Conference Talkscited by 5
  9. 2024M3AV: A Multimodal, Multigenre, and Multipurpose Audio-Visual Academic Lecture Datasetcited by 4
  10. 2023Knowledge-Aware Audio-Grounded Generative Slot Filling for Limited Annotated Datacited by 3
  11. 2025Zero-shot Context Biasing with Trie-based Decoding using Synthetic Multi-Pronunciationcited by 3
  12. 2024wav2graph: A Framework for Supervised Learning Knowledge Graph from Speechcited by 1
5 more
  1. 2025PMF-CEC: Phoneme-augmented Multimodal Fusion for Context-aware ASR Error Correction with Error-specific Selective Decodingcited by 1
  2. 2025CMT-LLM: Contextual Multi-Talker ASR Utilizing Large Language Modelscited by 1
  3. 2025Improving Contextual ASR via Multi-grained Fusion with Large Language Modelscited by 1
  4. 2026OCR-Enhanced Multimodal ASR Can Read While Listeningcited by 1
  5. 2025PARCO: Phoneme-Augmented Robust Contextual ASR via Contrastive Entity Disambiguationcited by 0