# Convex Online Video Frame Subset Selection using Multiple Criteria for Data Efficient Autonomous Driving

arXiv: https://arxiv.org/abs/2103.13021 · HTML: https://mlsanity.com/paper/2103.13021/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 26 references that are in the map.
- First posted on arXiv 24 Mar 2021.

## Builds on (most cited first, 12 of 12)

- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [CVXPY: A Python-Embedded Modeling Language for Convex Optimization](https://mlsanity.com/paper/1603.00943/) (2016)
- [End-to-end Driving via Conditional Imitation Learning](https://mlsanity.com/paper/1710.02410/) (2017)
- [End-to-end Learning of Driving Models from Large-scale Video Datasets](https://mlsanity.com/paper/1612.01079/) (2016)
- [AdaFrame: Adaptive Frame Selection for Fast Video Recognition](https://mlsanity.com/paper/1811.12432/) (2018)
- [CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-driving](https://mlsanity.com/paper/1807.03776/) (2018)
- [Conditional Affordance Learning for Driving in Urban Environments](https://mlsanity.com/paper/1806.06498/) (2018)
- [Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car](https://mlsanity.com/paper/1704.07911/) (2017)
- [Multi-Agent Reinforcement Learning Based Frame Sampling for Effective Untrimmed Video Recognition](https://mlsanity.com/paper/1907.13369/) (2019)
- [Cost-Effective Training of Deep CNNs with Active Model Adaptation](https://mlsanity.com/paper/1802.05394/) (2018)
- [FFNet: Video Fast-Forwarding via Reinforcement Learning](https://mlsanity.com/paper/1805.02792/) (2018)

## Abstract (arXiv)

Training vision-based Urban Autonomous driving models is a challenging problem, which is highly researched in recent times. Training such models is a data-intensive task requiring the storage and processing of vast volumes of (possibly redundant) driving video data. In this paper, we study the problem of developing data-efficient autonomous driving systems. In this context, we study the problem of multi-criteria online video frame subset selection. We study convex optimization-based solutions and show that they are unable to provide solutions with high weightage to the loss of selected video frames. We design a novel convex optimization-based multi-criteria online subset selection algorithm that uses a thresholded concave function of selection variables. We also propose and study a submodular optimization-based algorithm. Extensive experiments using the driving simulator CARLA show that we are able to drop 80% of the frames while succeeding to complete 100% of the episodes w.r.t. the model trained on 100% data, in the most difficult task of taking turns. This results in a training time of less than 30% compared to training on the whole dataset. We also perform detailed experiments on prediction performances of various affordances used by the Conditional Affordance Learning (CAL) model and show that our subset selection improves performance on the crucial affordance "Relative Angle" during turns.