# Dynamic Face Video Segmentation via Reinforcement Learning

arXiv: https://arxiv.org/abs/1907.01296 · HTML: https://mlsanity.com/paper/1907.01296/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jul 2019.

## Built on by (most cited first, 10 of 10)

- [RoI Tanh-polar Transformer Network for Face Parsing in the Wild](https://mlsanity.com/paper/2102.02717/) (2021)
- [FP-Age: Leveraging Face Parsing Attention for Facial Age Estimation in the Wild](https://mlsanity.com/paper/2106.11145/) (2021)
- [A Reinforcement-Learning-Based Energy-Efficient Framework for Multi-Task Video Analytics Pipeline](https://mlsanity.com/paper/2104.04443/) (2021)
- [Reinforcement Learning for Block Decomposition of CAD Models](https://mlsanity.com/paper/2302.11066/) (2023)
- [Do Smart Glasses Dream of Sentimental Visions? Deep Emotionship Analysis for Eyewear Devices](https://mlsanity.com/paper/2201.09933/) (2022)
- [Efficient Video Segmentation Models with Per-frame Inference](https://mlsanity.com/paper/2202.12427/) (2022)
- [MemX: An Attention-Aware Smart Eyewear System for Personalized Moment Auto-capture](https://mlsanity.com/paper/2105.00916/) (2021)
- [Reinforced Pedestrian Attribute Recognition with Group Optimization Reward](https://mlsanity.com/paper/2205.14042/) (2022)
- [Distortion-Aware Network Pruning and Feature Reuse for Real-time Video Segmentation](https://mlsanity.com/paper/2206.09604/) (2022)
- [Dynamic-Aware Video Distillation: Optimizing Temporal Resolution Based on Video Semantics](https://mlsanity.com/paper/2506.02021/) (2025)

## Abstract (arXiv)

For real-time semantic video segmentation, most recent works utilised a dynamic framework with a key scheduler to make online key/non-key decisions. Some works used a fixed key scheduling policy, while others proposed adaptive key scheduling methods based on heuristic strategies, both of which may lead to suboptimal global performance. To overcome this limitation, we model the online key decision process in dynamic video segmentation as a deep reinforcement learning problem and learn an efficient and effective scheduling policy from expert information about decision history and from the process of maximising global return. Moreover, we study the application of dynamic video segmentation on face videos, a field that has not been investigated before. By evaluating on the 300VW dataset, we show that the performance of our reinforcement key scheduler outperforms that of various baselines in terms of both effective key selections and running speed. Further results on the Cityscapes dataset demonstrate that our proposed method can also generalise to other scenarios. To the best of our knowledge, this is the first work to use reinforcement learning for online key-frame decision in dynamic video segmentation, and also the first work on its application on face videos.