mlsanityData to 14 Oct 2026

Regularizing Action Policies for Smooth Control with Reinforcement Learning

arXiv 2012.06644 ↗First posted 11 Dec 2020cs.RO cs.LG cs.SYcited by 59

Not on the map. Why?

Cited by 30 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

A critical problem with the practical utility of controllers trained with deep Reinforcement Learning (RL) is the notable lack of smoothness in the actions learned by the RL policies. This trend often presents itself in the form of control signal oscillation and can result in poor control, high power consumption, and undue system wear. We introduce Conditioning for Action Policy Smoothness (CAPS), an effective yet intuitive regularization on action policies, which offers consistent improvement in the smoothness of the learned state-to-action mappings of neural network controllers, reflected in the elimination of high-frequency components in the control signal. Tested on a real system, improvements in controller smoothness on a quadrotor drone resulted in an almost 80% reduction in power consumption while consistently training flight-worthy controllers. Project website: http://ai.bu.edu/caps

Key facts. Cited by 59 papers in the map (data through 14 Oct 2026). First posted on arXiv 11 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 4; 2023: 8; 2024: 10; 2025: 9; 2026: 25.

59 papers.

Sort:

By area:

  1. 2022L2C2: Locally Lipschitz Continuous Constraint towards Stable and Smooth Reinforcement Learningcited by 27
  2. 2021Latent Imagination Facilitates Zero-Shot Transfer in Autonomous Racingcited by 26
  3. 2025The Reality Gap in Robotics: Challenges, Solutions, and Best Practicescited by 24
  4. 2023Benchmarking Offline Reinforcement Learning on Real-Robot Hardwarecited by 23
  5. 2022Delayed Reinforcement Learning by Imitationcited by 16
  6. 2025Action Tokenizer Matters in In-Context Imitation Learningcited by 14
  7. 2022Inferring Smooth Control: Monte Carlo Posterior Policy Iteration with Gaussian Processescited by 10
  8. 2024Gradient-based Regularization for Action Smoothness in Robotic Control with Reinforcement Learningcited by 10
  9. 2025SkyDreamer: Interpretable End-to-End Vision-Based Drone Racing with Model-Based Reinforcement Learningcited by 8
  10. 2024Growing Q-Networks: Solving Continuous Control Tasks with Adaptive Control Resolutioncited by 7
  11. 2024Benchmarking Smoothness and Reducing High-Frequency Oscillations in Continuous Control Policiescited by 7
  12. 2021Data-Efficient Deep Reinforcement Learning for Attitude Control of Fixed-Wing UAVs: Field Experimentscited by 6
38 more
  1. 2023Identifying Expert Behavior in Offline Training Datasets Improves Behavioral Cloning of Robotic Manipulation Policiescited by 5
  2. 2024Legged Robot State Estimation With Invariant Extended Kalman Filter Using Neural Measurement Networkcited by 4
  3. 2026AGILE: A Comprehensive Workflow for Humanoid Loco-Manipulation Learningcited by 4
  4. 2022Policy Regularization for Legible Behaviorcited by 3
  5. 2023Real Robot Challenge 2022: Learning Dexterous Manipulation from Offline Data in the Real Worldcited by 3
  6. 2025TACO: General Acrobatic Flight Control via Target-and-Command-Oriented Reinforcement Learningcited by 3
  7. 2025COLSON: Controllable Learning-Based Social Navigation via Diffusion-Based Reinforcement Learningcited by 3
  8. 2025Learning Human-Like RL Agents Through Trajectory Optimization With Action Quantizationcited by 3
  9. 2026Enhancing Control Policy Smoothness by Aligning Actions with Predictions from Preceding Statescited by 3
  10. 2025Arnold: A multi-task, multi-embodiment muscle transformer policycited by 2
  11. 2026Stabilizing the Q-Gradient Field for Policy Smoothness in Actor-Critic Methodscited by 2
  12. 2026Learning Smooth Time-Varying Linear Policies with an Action Jacobian Penaltycited by 2
  13. 2021Decentralized Global Connectivity Maintenance for Multi-Robot Navigation: A Reinforcement Learning Approachcited by 1
  14. 2023Reinforcement Learning-Based Air Traffic Deconflictioncited by 1
  15. 2023Sense, Imagine, Act: Multimodal Perception Improves Model-Based Reinforcement Learning for Head-to-Head Autonomous Racingcited by 1
  16. 2023Towards Transferring Tactile-based Continuous Force Control Policies from Simulation to Robotcited by 1
  17. 2023Partial End-to-end Reinforcement Learning for Robustness Against Modelling Error in Autonomous Racingcited by 1
  18. 2024Explainable AI for Ship Collision Avoidance: Decoding Decision-Making Processes and Behavioral Intentionscited by 1
  19. 2024Let Hybrid A* Path Planner Obey Traffic Rules: A Deep Reinforcement Learning-Based Planning Frameworkcited by 1
  20. 2024Deep deterministic policy gradient with symmetric data augmentation for lateral attitude tracking control of a fixed-wing aircraftcited by 1
  21. 2024HG2P: Hippocampus-inspired High-reward Graph and Model-Free Q-Gradient Penalty for Path Planning and Motion Controlcited by 1
  22. 2026Efficient Knowledge Transfer for Jump-Starting Control Policy Learning of Multirotors through Physics-Aware Neural Architecturescited by 1
  23. 2026TaCarla: A comprehensive benchmarking dataset for end-to-end autonomous drivingcited by 1
  24. 2026A Hierarchical Spatiotemporal Action Tokenizer for In-Context Imitation Learning in Roboticscited by 1
  25. 2026LIDAR-AD: A Decoder-Free Latent-Interaction Dreamer with Action-Residual Chains for Autonomous Drivingcited by 1
  26. 2023Sim-Anchored Learning for On-the-Fly Adaptationcited by 0
  27. 2024Planning the path with Reinforcement Learning: Optimal Robot Motion Planning in RoboCup Small Size League Environmentscited by 0
  28. 2024Model-Free versus Model-Based Reinforcement Learning for Fixed-Wing UAV Attitude Control Under Varying Wind Conditionscited by 0
  29. 2025Deep Reinforcement Learning for Day-to-day Dynamic Tolling in Tradable Credit Schemescited by 0
  30. 2025Temporally smoothed incremental model-based heuristic dynamic programming for command-filtered cascaded online learning flight controlcited by 0
  31. 2026End-to-end example-based sim-to-real RL policy transfer based on neural stylisation with application to robotic cuttingcited by 0
  32. 2026CMR: Contractive Mapping Embeddings for Robust Humanoid Locomotion on Unstructured Terrainscited by 0
  33. 2026AnyUser: Translating Sketched User Intent into Domestic Robotscited by 0
  34. 2026Input-Side Variance Suppression under Non-Normal Transient Amplification in Continuous-Control Reinforcement Learningcited by 0
  35. 2026ARC-RL: A Reinforcement Learning Playground Inspired by ARC Raiderscited by 0
  36. 2026Implicit Action Chunking for Smooth Continuous Controlcited by 0
  37. 2026ZAPS-DA: Zero-Phase Action Policy Smoothing with Decoupled Actor for Continuous Control in Reinforcement Learningcited by 0
  38. 2026Enhancing Human-Likeness in Reinforcement Learning Agents via Hierarchical Macro Action Quantizationcited by 0

50 most cited of 59 All 59 (JSON)