# Meta-Learning for Multi-objective Reinforcement Learning

arXiv: https://arxiv.org/abs/1811.03376 · HTML: https://mlsanity.com/paper/1811.03376/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Nov 2018.

## Built on by (most cited first, 30 of 34)

- [A Practical Guide to Multi-Objective Reinforcement Learning and Planning](https://mlsanity.com/paper/2103.09568/) (2021)
- [PD-MORL: Preference-Driven Multi-Objective Reinforcement Learning Algorithm](https://mlsanity.com/paper/2208.07914/) (2022)
- [Multi-Objective Meta Learning](https://mlsanity.com/paper/2102.07121/) (2021)
- [Scaling Pareto-Efficient Decision Making Via Offline Multi-Objective RL](https://mlsanity.com/paper/2305.00567/) (2023)
- [Meta-learning approaches for few-shot learning: A survey of recent advances](https://mlsanity.com/paper/2303.07502/) (2023)
- [Redeeming Intrinsic Rewards via Constrained Optimization](https://mlsanity.com/paper/2211.07627/) (2022)
- [Meta-Learning-Based Deep Reinforcement Learning for Multiobjective Optimization Problems](https://mlsanity.com/paper/2105.02741/) (2021)
- [Pareto Set Learning for Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2501.06773/) (2025)
- [Physics-Based Motion Imitation with Adversarial Differential Discriminators](https://mlsanity.com/paper/2505.04961/) (2025)
- [A First-Order Multi-Gradient Algorithm for Multi-Objective Bi-Level Optimization](https://mlsanity.com/paper/2401.09257/) (2024)
- [Learning Pareto Set for Multi-Objective Continuous Robot Control](https://mlsanity.com/paper/2406.18924/) (2024)
- [MO-MIX: Multi-Objective Multi-Agent Cooperative Decision-Making With Deep Reinforcement Learning](https://mlsanity.com/paper/2603.00730/) (2026)
- [Eliciting User Preferences for Personalized Multi-Objective Decision Making through Comparative Feedback](https://mlsanity.com/paper/2302.03805/) (2023)
- [An Offline Adaptation Framework for Constrained Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2409.09958/) (2024)
- [Human-in-the-Loop Policy Optimization for Preference-Based Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2401.02160/) (2024)
- [Utility-Based Reinforcement Learning: Unifying Single-objective and Multi-objective Reinforcement Learning](https://mlsanity.com/paper/2402.02665/) (2024)
- [C-MORL: Multi-Objective Reinforcement Learning through Efficient Discovery of Pareto Front](https://mlsanity.com/paper/2410.02236/) (2024)
- [MOORL: A Framework for Integrating Offline-Online Reinforcement Learning](https://mlsanity.com/paper/2506.09574/) (2025)
- [Non-orthogonal Age-Optimal Information Dissemination in Vehicular Networks: A Meta Multi-Objective Reinforcement Learning Approach](https://mlsanity.com/paper/2402.12260/) (2024)
- [PA2D-MORL: Pareto Ascent Directional Decomposition based Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2603.19579/) (2026)
- [Evaluating the progress of Deep Reinforcement Learning in the real world: aligning domain-agnostic and domain-specific research](https://mlsanity.com/paper/2107.03015/) (2021)
- [Latent-Conditioned Policy Gradient for Multi-Objective Deep Reinforcement Learning](https://mlsanity.com/paper/2303.08909/) (2023)
- [MFC-EQ: Mean-Field Control with Envelope Q-Learning for Moving Decentralized Agents in Formation](https://mlsanity.com/paper/2410.12062/) (2024)
- [PRISM: Parallel Reward Integration with Symmetry for MORL](https://mlsanity.com/paper/2602.18277/) (2026)
- [MIRACL: A Diverse Meta-Reinforcement Learning for Multi-Objective Multi-Echelon Combinatorial Supply Chain Optimisation](https://mlsanity.com/paper/2603.05760/) (2026)
- [Monolithic vs. hybrid controller for multi-objective Sim-to-Real learning](https://mlsanity.com/paper/2108.07514/) (2021)
- [A Scale-Independent Multi-Objective Reinforcement Learning with Convergence Analysis](https://mlsanity.com/paper/2302.04179/) (2023)
- [UCB-driven Utility Function Search for Multi-objective Reinforcement Learning](https://mlsanity.com/paper/2405.00410/) (2024)
- [Complex Instruction Following with Diverse Style Policies in Football Games](https://mlsanity.com/paper/2511.19885/) (2025)
- [Post Hoc Extraction of Pareto Fronts for Continuous Control](https://mlsanity.com/paper/2603.02628/) (2026)

## Abstract (arXiv)

Multi-objective reinforcement learning (MORL) is the generalization of standard reinforcement learning (RL) approaches to solve sequential decision making problems that consist of several, possibly conflicting, objectives. Generally, in such formulations, there is no single optimal policy which optimizes all the objectives simultaneously, and instead, a number of policies has to be found each optimizing a preference of the objectives. In other words, the MORL is framed as a meta-learning problem, with the task distribution given by a distribution over the preferences. We demonstrate that such a formulation results in a better approximation of the Pareto optimal solutions in terms of both the optimality and the computational efficiency. We evaluated our method on obtaining Pareto optimal policies using a number of continuous control problems with high degrees of freedom.