mlsanityData to 14 Oct 2026

Attention2Probability: Attention-Driven Terminology Probability Estimation for Robust Speech-to-Text System

arXiv 2508.18701 ↗First posted 26 Aug 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent advances in speech large language models (SLMs) have improved speech recognition and translation in general domains, but accurately generating domain-specific terms or neologisms remains challenging. To address this, we propose Attention2Probability: attention-driven terminology probability estimation for robust speech-to-text system, which is lightweight, flexible, and accurate. Attention2Probability converts cross-attention weights between speech and terminology into presence probabilities, and it further employs curriculum learning to enhance retrieval accuracy. Furthermore, to tackle the lack of data for speech-to-text tasks with terminology intervention, we create and release a new speech dataset with terminology to support future research in this area. Experimental results show that Attention2Probability significantly outperforms the VectorDB method on our test set. Specifically, its maximum recall rates reach 92.57% for Chinese and 86.83% for English. This high recall is achieved with a latency of only 8.71ms per query. Intervening in SLMs' recognition and translation tasks using Attention2Probability-retrieved terms improves terminology accuracy by 6-17%, while revealing that the current utilization of terminology by SLMs has limitations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 37 references that are in the map. First posted on arXiv 26 Aug 2025.

Builds on

22 of 37 references.

References · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  4. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  5. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  6. 2016Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learningcited by 257
  7. 2017Lexically Constrained Decoding for Sequence Generation Using Grid Beam Searchcited by 237
  8. 2018Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translationcited by 182
  9. 2021Few-NERD: A Few-Shot Named Entity Recognition Datasetcited by 152
  10. 2018AISHELL-2: Transforming Mandarin ASR Research Into Industrial Scalecited by 128
  11. 2021Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusioncited by 101
  12. 2023SALM: Speech-augmented Language Model with In-context Learning for Speech Recognition and Translationcited by 76
10 more
  1. 2019Training Neural Machine Translation To Apply Terminology Constraintscited by 73
  2. 2021Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoderscited by 48
  3. 2025MegaTTS 3: Sparse Alignment Enhanced Latent Diffusion Transformer for Zero-Shot Speech Synthesiscited by 41
  4. 2018Neural Machine Translation Decoding with Terminology Constraintscited by 36
  5. 2016SYSTRAN's Pure Neural Machine Translation Systemscited by 24
  6. 2021Facilitating Terminology Translation with Target Lemma Annotationscited by 20
  7. 2023PromptASR for contextualized ASR with controllable stylecited by 19
  8. 2025BR-ASR: Efficient and Scalable Bias Retrieval Framework for Contextual Biasing ASR in Speech LLMcited by 11
  9. 2025Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translationcited by 8
  10. 2025Locate-and-Focus: Enhancing Terminology Translation in Speech Language Modelscited by 3

Built on by

Not cited yet.