mlsanityData to 7 Oct 2026

MuZero with Self-competition for Rate Control in VP9 Video Compression

arXiv 2202.06626 ↗First posted 14 Feb 2022eess.IV cs.CV cs.LGcited by 27

Not on the map. Why?

Abstract (arXiv)

Video streaming usage has seen a significant rise as entertainment, education, and business increasingly rely on online video. Optimizing video compression has the potential to increase access and quality of content to users, and reduce energy use and costs overall. In this paper, we present an application of the MuZero algorithm to the challenge of video compression. Specifically, we target the problem of learning a rate control policy to select the quantization parameters (QP) in the encoding process of libvpx, an open source VP9 video compression library widely used by popular video-on-demand (VOD) services. We treat this as a sequential decision making problem to maximize the video quality with an episodic constraint imposed by the target bitrate. Notably, we introduce a novel self-competition based reward mechanism to solve constrained RL with variable constraint satisfaction difficulty, which is challenging for existing constrained RL methods. We demonstrate that the MuZero-based rate control achieves an average 6.28% reduction in size of the compressed videos for the same delivered video quality level (measured as PSNR BD-rate) compared to libvpx's two-pass VBR rate control policy, while having better constraint satisfaction behavior.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 25 of its 49 references that are in the map. First posted on arXiv 14 Feb 2022.

Builds on

25 of 49 references.

References · 25

  1. 2016Layer Normalizationcited by 5,579
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  3. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  4. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  5. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  6. 2017Constrained Policy Optimizationcited by 928
  7. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  8. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 555
  9. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 464
  10. 2018Reward Constrained Policy Optimizationcited by 397
  11. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 362
  12. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 278
13 more
  1. 2018Distributed Prioritized Experience Replaycited by 260
  2. 2020Exploration-Exploitation in Constrained MDPscited by 146
  3. 2019Constrained Reinforcement Learning Has Zero Duality Gapcited by 144
  4. 2019YouTube UGC Dataset for Video Compression Researchcited by 124
  5. 2016A Deep Hierarchical Approach to Lifelong Learning in Minecraftcited by 90
  6. 2020Chip Placement with Deep Reinforcement Learningcited by 83
  7. 2019Value constrained model-free continuous controlcited by 34
  8. 2018Ranked Reward: Enabling Self-Play Reinforcement Learning for Combinatorial Optimizationcited by 25
  9. 2020Balancing Constraints and Rewards with Meta-Gradient D4PGcited by 18
  10. 2020Reward Constrained Interactive Recommendation with Natural Language Feedbackcited by 6
  11. 2021A Dual-Critic Reinforcement Learning Framework for Frame-level Bit Allocation in HEVC/H.265cited by 5
  12. 2019Self-Play Learning Without a Reward Metriccited by 3
  13. 2020Neural Rate Control for Video Encoding using Imitation Learningcited by 2

Built on by

Cited by year

Citing papers per year: 2022: 7; 2023: 12; 2024: 5; 2025: 3.

27 papers.

Sort:

By area:

  1. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 169
  2. 2023Loss of Plasticity in Continual Deep Reinforcement Learningcited by 118
  3. 2022Adversarial Training for High-Stakes Reliabilitycited by 60
  4. 2022Evaluating Long-Term Memory in 3D Mazescited by 41
  5. 2023Last-Iterate Convergent Policy Gradient Primal-Dual Methods for Constrained MDPscited by 27
  6. 2023Beyond Games: A Systematic Review of Neural Monte Carlo Tree Search Applicationscited by 16
  7. 2023Diversifying AI: Towards Creative Chess with AlphaZerocited by 15
  8. 2023GVFs in the Real World: Making Predictions Online for Water Treatmentcited by 13
  9. 2022Fast and Precise: Adjusting Planning Horizon with Adaptive Subgoal Searchcited by 12
  10. 2022Epistemic Monte Carlo Tree Searchcited by 7
  11. 2023What model does MuZero learn?cited by 7
  12. 2023Hybrid Search for Efficient Planning with Completeness Guaranteescited by 6
15 more
  1. 2022Reinforcement Learning with a Terminatorcited by 4
  2. 2023Equivariant MuZerocited by 3
  3. 2023Policy-Based Self-Competition for Planning Problemscited by 3
  4. 2022Policy Gradient With Serial Markov Chain Reasoningcited by 2
  5. 2023Real-time scheduling of renewable power systems through planning-based reinforcement learningcited by 2
  6. 2023Deep Video Codec Control for Vision Modelscited by 2
  7. 2024Multi-Agent Reinforcement Learning for Maritime Operational Technology Cyber Securitycited by 2
  8. 2025RL-RC-DoT: A Block-level RL agent for Task-Aware Video Compressioncited by 2
  9. 2025OptionZero: Planning with Learned Optionscited by 2
  10. 2024ReACT: Reinforcement Learning for Controller Parametrization using B-Spline Geometriescited by 1
  11. 2024A Perspective on Deep Vision Performance with Standard Image and Video Codecscited by 1
  12. 2024Demystifying MuZero Planning: Interpreting the Learned Modelcited by 1
  13. 2023Agents Explore the Environment Beyond Good Actions to Improve Their Model for Better Decisionscited by 0
  14. 2024Deterministic Policy Gradient Primal-Dual Methods for Continuous-Space Constrained MDPscited by 0
  15. 2025Resource-Aware Neural Network Pruning Using Graph-based Reinforcement Learningcited by 0