# Straight to the Point: Fast-forwarding Videos via Reinforcement Learning Using Textual Data

arXiv: https://arxiv.org/abs/2003.14229 · HTML: https://mlsanity.com/paper/2003.14229/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Mar 2020.

## Built on by (most cited first, 5 of 5)

- [Collaborative Multi-Agent Video Fast-Forwarding](https://mlsanity.com/paper/2305.17569/) (2023)
- [Zeus: Efficiently Localizing Actions in Videos using Reinforcement Learning](https://mlsanity.com/paper/2104.06142/) (2021)
- [Text-Driven Video Acceleration: A Weakly-Supervised Reinforcement Learning Method](https://mlsanity.com/paper/2203.15778/) (2022)
- [On the impact of MDP design for Reinforcement Learning agents in Resource Management](https://mlsanity.com/paper/2109.03202/) (2021)
- [TRIMMER: A New Paradigm for Video Summarization through Self-Supervised Reinforcement Learning](https://mlsanity.com/paper/2605.01659/) (2026)

## Abstract (arXiv)

The rapid increase in the amount of published visual data and the limited time of users bring the demand for processing untrimmed videos to produce shorter versions that convey the same information. Despite the remarkable progress that has been made by summarization methods, most of them can only select a few frames or skims, which creates visual gaps and breaks the video context. In this paper, we present a novel methodology based on a reinforcement learning formulation to accelerate instructional videos. Our approach can adaptively select frames that are not relevant to convey the information without creating gaps in the final video. Our agent is textually and visually oriented to select which frames to remove to shrink the input video. Additionally, we propose a novel network, called Visually-guided Document Attention Network (VDAN), able to generate a highly discriminative embedding space to represent both textual and visual data. Our experiments show that our method achieves the best performance in terms of F1 Score and coverage at the video segment level.