mlsanityData to 14 Oct 2026

Incorporating External POS Tagger for Punctuation Restoration

arXiv 2106.06731 ↗First posted 12 Jun 2021cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Punctuation restoration is an important post-processing step in automatic speech recognition. Among other kinds of external information, part-of-speech (POS) taggers provide informative tags, suggesting each input token's syntactic role, which has been shown to be beneficial for the punctuation restoration task. In this work, we incorporate an external POS tagger and fuse its predicted labels into the existing language model to provide syntactic information. Besides, we propose sequence boundary sampling (SBS) to learn punctuation positions more efficiently as a sequence tagging task. Experimental results show that our methods can consistently obtain performance gains and achieve a new state-of-the-art on the common IWSLT benchmark. Further ablation studies illustrate that both large pre-trained language models and the external POS tagger take essential parts to improve the model's performance.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 11 of its 32 references that are in the map. First posted on arXiv 12 Jun 2021.

Builds on

11 of 32 references.

References · 11

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  6. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  7. 2020Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processingcited by 162
  8. 2020Controllable Time-Delay Transformer for Real-Time Punctuation Prediction and Disfluency Detectioncited by 23
  9. 2020Multimodal Semi-supervised Learning Framework for Punctuation Prediction in Conversational Speechcited by 18
  10. 2020Adversarial Transfer Learning for Punctuation Restorationcited by 15
  11. 2017Joint Learning of Correlated Sequence Labelling Tasks Using Bidirectional Recurrent Neural Networkscited by 6

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 3; 2023: 1.

5 papers.

  1. 2021BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamesecited by 34
  2. 2022A Context-Aware Feature Fusion Framework for Punctuation Restorationcited by 2
  3. 2022Text Editing as Imitation Gamecited by 1
  4. 2022FF2: A Feature Fusion Two-Stream Framework for Punctuation Restorationcited by 0
  5. 2023A Small and Fast BERT for Chinese Medical Punctuation Restorationcited by 0