Auxiliary Reward Generation with Transition Distance Representation Learning
On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation
Abstract (arXiv)
Reinforcement learning (RL) has shown its strength in challenging sequential decision-making problems. The reward function in RL is crucial to the learning performance, as it serves as a measure of the task completion degree. In real-world problems, the rewards are predominantly human-designed, which requires laborious tuning, and is easily affected by human cognitive biases. To achieve automatic auxiliary reward generation, we propose a novel representation learning approach that can measure the ``transition distance'' between states. Building upon these representations, we introduce an auxiliary reward generation technique for both single-task and skill-chaining scenarios without the need for human knowledge. The proposed approach is evaluated in a wide range of manipulation tasks. The experiment results demonstrate the effectiveness of measuring the transition distance between states and the induced improvement by auxiliary rewards, which not only promotes better learning efficiency but also increases convergent stability.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 20 of its 37 references that are in the map. First posted on arXiv 12 Feb 2024.
Builds on
20 of 37 references.
References · 20
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
- 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
- 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
- 2023Mastering Diverse Domains through World Modelscited by 1,253
- 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
- 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 673
- 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 511
- 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 274
- 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 231
- 2015Contextual Markov Decision Processescited by 220
8 more
- 2020Learning to Utilize Shaping Rewards: A New Approach of Reward Shapingcited by 114
- 2022Factory: Fast Contact for Robotic Assemblycited by 54
- 2021Adversarial Skill Chaining for Long-Horizon Robot Manipulation via Terminal State Regularizationcited by 46
- 2015From Pixels to Torques: Policy Learning with Deep Dynamical Modelscited by 45
- 2019Hierarchical Reinforcement Learning with Advantage-Based Auxiliary Rewardscited by 40
- 2022Mask-based Latent Reconstruction for Reinforcement Learningcited by 39
- 2019Reward Shaping via Meta-Learningcited by 32
- 2021Training Transition Policies via Distribution Matching for Complex Taskscited by 6
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Generating Auxiliary Tasks with Reinforcement Learning
- 2025Reward Training Wheels: Adaptive Auxiliary Rewards for Robotics Reinforcement Learning
- 2026ARM: Advantage Reward Modeling for Long-Horizon Manipulation
- 2024Reinforcement Learning via Auxiliary Task Distillation
- 2026Large Reward Models: Generalizable Online Robot Reward Generation with Vision-Language Models
- 2026RARM: Confidence-Gated Progress Reward Modeling for RL in Manipulation
- 2026Disentangling Task-Relevant Latent Dynamics for Reward Modeling
- 2026Causal Reward World Models: Zero-shot Reward Design for Automated Skill Generation