# The importance of fillers for text representations of speech transcripts

arXiv: https://arxiv.org/abs/2009.11340 · HTML: https://mlsanity.com/paper/2009.11340/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Sep 2020.

## Built on by (most cited first, 15 of 15)

- [Automatic Text Evaluation through the Lens of Wasserstein Barycenters](https://mlsanity.com/paper/2108.12463/) (2021)
- [A Novel Estimator of Mutual Information for Learning to Disentangle Textual Representations](https://mlsanity.com/paper/2105.02685/) (2021)
- [InfoLM: A New Metric to Evaluate Summarization & Data2Text Generation](https://mlsanity.com/paper/2112.01589/) (2021)
- [Rainproof: An Umbrella To Shield Text Generators From Out-Of-Distribution Data](https://mlsanity.com/paper/2212.09171/) (2022)
- [Improving Multimodal fusion via Mutual Dependency Maximisation](https://mlsanity.com/paper/2109.00922/) (2021)
- [Unsupervised Layer-wise Score Aggregation for Textual OOD Detection](https://mlsanity.com/paper/2302.09852/) (2023)
- [Learning Disentangled Textual Representations via Statistical Measures of Similarity](https://mlsanity.com/paper/2205.03589/) (2022)
- [Towards More Robust NLP System Evaluation: Handling Missing Scores in Benchmarks](https://mlsanity.com/paper/2305.10284/) (2023)
- [Beam Search with Bidirectional Strategies for Neural Response Generation](https://mlsanity.com/paper/2110.03389/) (2021)
- [Code-switched inspired losses for generic spoken dialog representations](https://mlsanity.com/paper/2108.12465/) (2021)
- [Fillers in Spoken Language Understanding: Computational and Psycholinguistic Perspectives](https://mlsanity.com/paper/2301.10761/) (2023)
- [Beyond Mahalanobis-Based Scores for Textual OOD Detection](https://mlsanity.com/paper/2211.13527/) (2022)
- [LiveLongBench: Tackling Long-Context Understanding for Spoken Texts from Live Streams](https://mlsanity.com/paper/2504.17366/) (2025)
- [A Novel Information-Theoretic Objective to Disentangle Representations for Fair Classification](https://mlsanity.com/paper/2310.13990/) (2023)
- [What are the best systems? New perspectives on NLP Benchmarking](https://mlsanity.com/paper/2202.03799/) (2022)

## Abstract (arXiv)

While being an essential component of spoken language, fillers (e.g."um" or "uh") often remain overlooked in Spoken Language Understanding (SLU) tasks. We explore the possibility of representing them with deep contextualised embeddings, showing improvements on modelling spoken language and two downstream tasks - predicting a speaker's stance and expressed confidence.