mlsanityData to 14 Oct 2026

Joint Automatic Speech Recognition And Structure Learning For Better Speech Understanding

arXiv 2501.07329 ↗First posted 13 Jan 2025cs.SD cs.CL eess.AScited by 0

Not on the map. Why?

Abstract (arXiv)

Spoken language understanding (SLU) is a structure prediction task in the field of speech. Recently, many works on SLU that treat it as a sequence-to-sequence task have achieved great success. However, This method is not suitable for simultaneous speech recognition and understanding. In this paper, we propose a joint speech recognition and structure learning framework (JSRSL), an end-to-end SLU model based on span, which can accurately transcribe speech and extract structured content simultaneously. We conduct experiments on name entity recognition and intent classification using the Chinese dataset AISHELL-NER and the English dataset SLURP. The results show that our proposed method not only outperforms the traditional sequence-to-sequence method in both transcription and extraction capabilities but also achieves state-of-the-art performance on the two datasets.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 36 references that are in the map. First posted on arXiv 13 Jan 2025.

Builds on

29 of 36 references.

References · 29

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  4. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 601
  5. 2020SLURP: A Spoken Language Understanding Resource Packagecited by 198
  6. 2022Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognitioncited by 144
  7. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 140
  8. 2019Semantics-aware BERT for Language Understandingcited by 112
  9. 2023FunASR: A Fundamental End-to-End Speech Recognition Toolkitcited by 95
  10. 2021SLUE: New Benchmark Tasks for Spoken Language Understanding Evaluation on Natural Speechcited by 72
  11. 2023Efficient Sequence Transduction by Jointly Predicting Tokens and Durationscited by 58
  12. 2021SpanNER: Named Entity Re-/Recognition as Span Predictioncited by 55
17 more
  1. 2021ESPnet-SLU: Advancing Spoken Language Understanding through ESPnetcited by 53
  2. 2020End-to-end Named Entity Recognition from English Speechcited by 36
  3. 2023BatGPT: A Bidirectional Autoregessive Talker from Generative Pre-trained Transformercited by 23
  4. 2019Dependency or Span, End-to-End Uniform Semantic Role Labelingcited by 22
  5. 2022AISHELL-NER: Named Entity Recognition from Chinese Speechcited by 19
  6. 2022A Study on the Integration of Pre-trained SSL, ASR, LM and SLU Models for Spoken Language Understandingcited by 18
  7. 2022Global Pointer: Novel Efficient Span-based Approach for Named Entity Recognitioncited by 15
  8. 2021Seeking Common but Distinguishing Difference, A Joint Aspect-based Sentiment Analysis Modelcited by 13
  9. 2021On the Use of External Data for Spoken Named Entity Recognitioncited by 12
  10. 2022Token-level Sequence Labeling for Spoken Language Understanding using Compositional End-to-End Modelscited by 10
  11. 2023UniverSLU: Universal Spoken Language Understanding for Diverse Tasks with Natural Language Instructionscited by 9
  12. 2022Context-aware Fine-tuning of Self-supervised Speech Modelscited by 7
  13. 2021On joint training with interfaces for spoken language understandingcited by 5
  14. 2021Sparse Fuzzy Attention for Structured Sentiment Analysiscited by 5
  15. 2022Introducing Semantics into Speech Encoderscited by 3
  16. 2023Generative Context-aware Fine-tuning of Self-supervised Speech Modelscited by 3
  17. 2023Leveraging Multilingual Self-Supervised Pretrained Models for Sequence-to-Sequence End-to-End Spoken Language Understandingcited by 1

Built on by

Not cited yet.