mlsanityData to 14 Oct 2026

Robotic Skill Diversification via Active Mutation of Reward Functions in Reinforcement Learning During a Liquid Pouring Task

arXiv 2509.18463 ↗First posted 22 Sep 2025cs.RO cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

This paper explores how deliberate mutations of reward function in reinforcement learning can produce diversified skill variations in robotic manipulation tasks, examined with a liquid pouring use case. To this end, we developed a new reward function mutation framework that is based on applying Gaussian noise to the weights of the different terms in the reward function. Inspired by the cost-benefit tradeoff model from human motor control, we designed the reward function with the following key terms: accuracy, time, and effort. The study was performed in a simulation environment created in NVIDIA Isaac Sim, and the setup included Franka Emika Panda robotic arm holding a glass with a liquid that needed to be poured into a container. The reinforcement learning algorithm was based on Proximal Policy Optimization. We systematically explored how different configurations of mutated weights in the rewards function would affect the learned policy. The resulting policies exhibit a wide range of behaviours: from variations in execution of the originally intended pouring task to novel skills useful for unexpected tasks, such as container rim cleaning, liquid mixing, and watering. This approach offers promising directions for robotic systems to perform diversified learning of specific tasks, while also potentially deriving meaningful skills for future tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 13 of its 43 references that are in the map. First posted on arXiv 22 Sep 2025.

Builds on

13 of 43 references.

References · 13

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 571
  3. 2015Illuminating search spaces by mapping elitescited by 425
  4. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
  5. 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 349
  6. 2018Residual Reinforcement Learning for Robot Controlcited by 274
  7. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 223
  8. 2020Reward Machines: Exploiting Reward Function Structure in Reinforcement Learningcited by 191
  9. 2018A survey on policy search algorithms for learning robot controllers in a handful of trialscited by 35
  10. 2024Vision-Language Model-based Physical Reasoning for Robot Liquid Perceptioncited by 10
  11. 2021Efficient and Reactive Planning for High Speed Robot Air Hockeycited by 8
  12. 2020Robotic Arm Control and Task Training through Deep Reinforcement Learningcited by 6
1 more
  1. 2025Visio-Verbal Teleimpedance Interface: Enabling Semi-Autonomous Control of Physical Interaction via Eye Tracking and Speechcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Efficient Language-instructed Skill Acquisition via Reward-Policy Co-Evolution
  2. 2025Automated Hybrid Reward Scheduling via Large Language Models for Robotic Skill Learning
  3. 2024Unsupervised Skill Discovery for Robotic Manipulation through Automatic Task Generation
  4. 2025ReinforceGen: Hybrid Skill Policies with Automated Data Generation and Reinforcement Learning
  5. 2024SLIM: Skill Learning with Multiple Critics
  6. 2026Beyond Pixels: Learning Invariant Rewards for Real-World Robotics From a Few Demonstrations
  7. 2025Robo-Dopamine: General Process Reward Modeling for High-Precision Robotic Manipulation
  8. 2026Coherent Off-Policy Improvement of Large Behavior Models with Learned Rewards