# Learning to Skim Text

arXiv: https://arxiv.org/abs/1704.06877 · HTML: https://mlsanity.com/paper/1704.06877/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Apr 2017.

## Built on by (most cited first, 18 of 18)

- [Dynamic Neural Networks: A Survey](https://mlsanity.com/paper/2102.04906/) (2021)
- [PonderNet: Learning to Ponder](https://mlsanity.com/paper/2107.05407/) (2021)
- [A Survey on Green Deep Learning](https://mlsanity.com/paper/2111.05193/) (2021)
- [Revisiting the Weaknesses of Reinforcement Learning for Neural Machine Translation](https://mlsanity.com/paper/2106.08942/) (2021)
- [A Survey on Dynamic Neural Networks for Natural Language Processing](https://mlsanity.com/paper/2202.07101/) (2022)
- [Transkimmer: Transformer Learns to Layer-wise Skim](https://mlsanity.com/paper/2205.07324/) (2022)
- [Block-Skim: Efficient Question Answering for Transformer](https://mlsanity.com/paper/2112.08560/) (2021)
- [Characterization of Time-variant and Time-invariant Assessment of Suicidality on Reddit using C-SSRS](https://mlsanity.com/paper/2104.04140/) (2021)
- [Internally Rewarded Reinforcement Learning](https://mlsanity.com/paper/2302.00270/) (2023)
- [Boosted Dynamic Neural Networks](https://mlsanity.com/paper/2211.16726/) (2022)
- [Doping: A technique for efficient compression of LSTM models using sparse structured additive matrices](https://mlsanity.com/paper/2102.07071/) (2021)
- [Part & Whole Extraction: Towards A Deep Understanding of Quantitative Facts for Percentages in Text](https://mlsanity.com/paper/2110.13505/) (2021)
- [State-Regularized Recurrent Neural Networks to Extract Automata and Explain Predictions](https://mlsanity.com/paper/2212.05178/) (2022)
- [Control Reinforcement Learning: Interpretable Token-Level Steering of LLMs via Sparse Autoencoder Features](https://mlsanity.com/paper/2602.10437/) (2026)
- [No-Skim: Towards Efficiency Robustness Evaluation on Skimming-based Language Models](https://mlsanity.com/paper/2312.09494/) (2023)
- [Informative Text Generation from Knowledge Triples](https://mlsanity.com/paper/2209.12733/) (2022)
- [SOI: Scaling Down Computational Complexity by Estimating Partial States of the Model](https://mlsanity.com/paper/2410.03813/) (2024)
- [Rethinking Recurrent Neural Networks for Time Series Forecasting: A Reinforced Recurrent Encoder with Prediction-Oriented Proximal Policy Optimization](https://mlsanity.com/paper/2601.03683/) (2026)

## Abstract (arXiv)

Recurrent Neural Networks are showing much promise in many sub-areas of natural language processing, ranging from document classification to machine translation to automatic question answering. Despite their promise, many recurrent models have to read the whole text word by word, making it slow to handle long documents. For example, it is difficult to use a recurrent network to read a book and answer questions about it. In this paper, we present an approach of reading text while skipping irrelevant information if needed. The underlying model is a recurrent network that learns how far to jump after reading a few words of the input text. We employ a standard policy gradient method to train the model to make discrete jumping decisions. In our benchmarks on four different tasks, including number prediction, sentiment analysis, news article classification and automatic Q&A, our proposed model, a modified LSTM with jumping, is up to 6 times faster than the standard sequential LSTM, while maintaining the same or even better accuracy.