Data to 7 Oct 2026

Hybrid Reward Architecture for Reinforcement Learning

arXiv 1706.04208 ↗First posted 13 Jun 2017cs.LGcited by 57

Not on the map. Why?

Abstract (arXiv)

One of the main challenges in reinforcement learning (RL) is generalisation. In typical deep RL methods this is achieved by approximating the optimal value function with a low-dimensional representation using a deep network. While this approach works well in many domains, in domains where the optimal value function cannot easily be reduced to a low-dimensional representation, learning can be very slow and unstable. This paper contributes towards tackling such challenging domains, by proposing a new method, called Hybrid Reward Architecture (HRA). HRA takes as input a decomposed reward function and learns a separate value function for each component reward function. Because each component typically only depends on a subset of all features, the corresponding value function can be approximated more easily by a low-dimensional representation, enabling more effective learning. We demonstrate HRA on a toy-problem and the Atari game Ms. Pac-Man, where HRA achieves above-human performance.

Key facts. Cited by 57 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Jun 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 15; 2023: 10; 2024: 9; 2025: 6; 2026: 5.

57 papers.

Sort:

By area:

2021Acquisition of Chess Knowledge in AlphaZerocited by 120
2021Reinforcement Learning, Bit by Bitcited by 59
2022Unpacking Reward Shaping: Understanding the Benefits of Reward Engineering on Sample Complexitycited by 59
2023Leveraging Factored Action Spaces for Efficient Offline Reinforcement Learning in Healthcarecited by 41
2022How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Compositioncited by 27
2021Distributional Reinforcement Learning for Multi-Dimensional Reward Functionscited by 21
2022Solving Continuous Control via Q-learningcited by 21
2022Reinforcement learning on graphs: A surveycited by 18
2021AoI-Aware Resource Allocation for Platoon-Based C-V2X Networks via Multi-Agent Multi-Task Reinforcement Learningcited by 17
2021Computational Benefits of Intermediate Rewards for Goal-Reaching Policy Learningcited by 17
2024Continuous Control with Coarse-to-fine Reinforcement Learningcited by 15
2023Explaining Reinforcement Learning Agents Through Counterfactual Action Outcomescited by 13

38 more

2022Integrating Policy Summaries with Reward Decomposition for Explaining Reinforcement Learning Agentscited by 10
2022Asynchronous Hybrid Reinforcement Learning for Latency and Reliability Optimization in the Metaverse over Wireless Communicationscited by 9
2023Residual Q-Learning: Offline and Online Policy Customization without Valuecited by 9
2022Distributional Reinforcement Learning with Regularized Wasserstein Losscited by 8
2022Reinforcement Learning Your Way: Agent Characterization through Policy Regularizationcited by 6
2022Value Function Decomposition for Iterative Design of Reinforcement Learning Agentscited by 6
2024Growing Q-Networks: Solving Continuous Control Tasks with Adaptive Control Resolutioncited by 6
2023Prioritized Soft Q-Decomposition for Lexicographic Reinforcement Learningcited by 5
2022Orchestrated Value Mapping for Reinforcement Learningcited by 4
2021Locality Matters: A Scalable Value Decomposition Approach for Cooperative Multi-Agent Reinforcement Learningcited by 3
2022Flexible Multiple-Objective Reinforcement Learning for Chip Placementcited by 3
2023Towards model-free RL algorithms that scale well with unstructured datacited by 3
2021A Genetic Algorithm with Tree-structured Mutation for Hyperparameter Optimisation of Graph Neural Networkscited by 2
2021Learning robust driving policies without online explorationcited by 2
2021Adapting to Reward Progressivity via Spectral Reinforcement Learningcited by 2
2021Decomposing the Prediction Problem; Autonomous Navigation by neoRL Agentscited by 2
2022Interpretable and Effective Reinforcement Learning for Attacking against Graph-based Rumor Detectioncited by 2
2022Modularity benefits reinforcement learning agents with competing homeostatic drivescited by 2
2024Reward-Punishment Reinforcement Learning with Maximum Entropycited by 2
2024Off-Policy Reinforcement Learning with High Dimensional Rewardcited by 2
2024Quasimetric Value Functions with Dense Rewardscited by 2
2025Automated Hybrid Reward Scheduling via Large Language Models for Robotic Skill Learningcited by 2
2025Dual-Objective Reinforcement Learning with Novel Hamilton-Jacobi-Bellman Formulationscited by 2
2021Time-Aware Q-Networks: Resolving Temporal Irregularity for Deep Reinforcement Learningcited by 1
2023User-centric Heterogeneous-action Deep Reinforcement Learning for Virtual Reality in the Metaverse over Wireless Networkscited by 1
2023On the Value of Myopic Behavior in Policy Reusecited by 1
2024Sequence Compression Speeds Up Credit Assignment in Reinforcement Learningcited by 1
2025Explainable Reinforcement Learning for Formula One Race Strategycited by 1
2025Strategy Masking: A Method for Guardrails in Value-based Reinforcement Learning Agentscited by 1
2021Compositional Q-learning for electrolyte repletion with imbalanced patient sub-populationscited by 0
2022Towards neoRL networks; the emergence of purposive graphscited by 0
2022Symbolic Explanation of Affinity-Based Reinforcement Learning Agents with Markov Modelscited by 0
2023Deep Reinforcement Learning for Asset Allocation: Reward Clippingcited by 0
2023Virtual Reality in Metaverse over Wireless Networks with User-centered Deep Reinforcement Learningcited by 0
2023Heterogeneous 360 Degree Videos in Metaverse: Differentiated Reinforcement Learning Approachescited by 0
2024Radio Resource Management and Path Planning in Intelligent Transportation Systems via Reinforcement Learning for Environmental Sustainabilitycited by 0
2024Segmenting Action-Value Functions Over Time-Scales in SARSA via TD(Delta)cited by 0
2024From approximation error to optimality gap -- Explaining the performance impact of opportunity cost approximation in integrated demand management and vehicle routingcited by 0

50 most cited of 57 All 57 (JSON)