# Towards Machine Comprehension of Spoken Content: Initial TOEFL Listening Comprehension Test by Machine

arXiv: https://arxiv.org/abs/1608.06378 · HTML: https://mlsanity.com/paper/1608.06378/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Aug 2016.

## Built on by (most cited first, 23 of 23)

- [L-Eval: Instituting Standardized Evaluation for Long Context Language Models](https://mlsanity.com/paper/2307.11088/) (2023)
- [A Survey on Large Language Model Acceleration based on KV Cache Management](https://mlsanity.com/paper/2412.19442/) (2024)
- [Scaling Laws of RoPE-based Extrapolation](https://mlsanity.com/paper/2310.05209/) (2023)
- [QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension](https://mlsanity.com/paper/2107.12708/) (2021)
- [SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Words](https://mlsanity.com/paper/2406.13340/) (2024)
- [End-to-end Spoken Conversational Question Answering: Task, Dataset and Model](https://mlsanity.com/paper/2204.14272/) (2022)
- [The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs](https://mlsanity.com/paper/2504.17768/) (2025)
- [Is It Really Long Context if All You Need Is Retrieval? Towards Genuinely Difficult Long Context NLP](https://mlsanity.com/paper/2407.00402/) (2024)
- [Dataset Decomposition: Faster LLM Training with Variable Sequence Length Curriculum](https://mlsanity.com/paper/2405.13226/) (2024)
- [HeySQuAD: A Spoken Question Answering Dataset](https://mlsanity.com/paper/2304.13689/) (2023)
- [DUAL: Discrete Spoken Unit Adaptive Learning for Textless Spoken Question Answering](https://mlsanity.com/paper/2203.04911/) (2022)
- [Aligning Source Visual and Target Language Domains for Unpaired Video Captioning](https://mlsanity.com/paper/2211.12148/) (2022)
- [LibriSQA: A Novel Dataset and Framework for Spoken Question Answering with Large Language Models](https://mlsanity.com/paper/2308.10390/) (2023)
- [Zebra: Extending Context Window with Layerwise Grouped Local-Global Attention](https://mlsanity.com/paper/2312.08618/) (2023)
- [SLIDE: Integrating Speech Language Model with LLM for Spontaneous Spoken Dialogue Generation](https://mlsanity.com/paper/2501.00805/) (2025)
- [SkyLadder: Better and Faster Pretraining via Context Window Scheduling](https://mlsanity.com/paper/2503.15450/) (2025)
- [Analyzing Multiple-Choice Reading and Listening Comprehension Tests](https://mlsanity.com/paper/2307.01076/) (2023)
- [Retrieval Or Holistic Understanding? Dolce: Differentiate Our Long Context Evaluation Tasks](https://mlsanity.com/paper/2409.06338/) (2024)
- [RSQ: Learning from Important Tokens Leads to Better Quantized LLMs](https://mlsanity.com/paper/2503.01820/) (2025)
- [Multi-Facet Blending for Faceted Query-by-Example Retrieval](https://mlsanity.com/paper/2412.01443/) (2024)
- [Audio-Oriented Multimodal Machine Comprehension: Task, Dataset and Model](https://mlsanity.com/paper/2107.01571/) (2021)
- [An Initial Investigation of Non-Native Spoken Question-Answering](https://mlsanity.com/paper/2107.04691/) (2021)
- [Long Input Benchmark for Russian Analysis](https://mlsanity.com/paper/2408.02439/) (2024)

## Abstract (arXiv)

Multimedia or spoken content presents more attractive information than plain text content, but it's more difficult to display on a screen and be selected by a user. As a result, accessing large collections of the former is much more difficult and time-consuming than the latter for humans. It's highly attractive to develop a machine which can automatically understand spoken content and summarize the key information for humans to browse over. In this endeavor, we propose a new task of machine comprehension of spoken content. We define the initial goal as the listening comprehension test of TOEFL, a challenging academic English examination for English learners whose native language is not English. We further propose an Attention-based Multi-hop Recurrent Neural Network (AMRNN) architecture for this task, achieving encouraging results in the initial tests. Initial results also have shown that word-level attention is probably more robust than sentence-level attention for this task with ASR errors.