# Multi-Objective Optimization Using Adaptive Distributed Reinforcement Learning

arXiv: https://arxiv.org/abs/2403.08879 · HTML: https://mlsanity.com/paper/2403.08879/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 15 of its 59 references that are in the map.
- First posted on arXiv 13 Mar 2024.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 15 of 15)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [A Practical Guide to Multi-Objective Reinforcement Learning and Planning](https://mlsanity.com/paper/2103.09568/) (2021)
- [Asynchronous Decentralized Parallel Stochastic Gradient Descent](https://mlsanity.com/paper/1710.06952/) (2017)
- [Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments](https://mlsanity.com/paper/1710.03641/) (2017)
- [Deep Reinforcement Learning for Intelligent Transportation Systems: A Survey](https://mlsanity.com/paper/2005.00935/) (2020)
- [Dif-MAML: Decentralized Multi-Agent Meta-Learning](https://mlsanity.com/paper/2010.02870/) (2020)
- [Multi-objective Reinforcement Learning with Continuous Pareto Frontier Approximation Supplementary Material](https://mlsanity.com/paper/1406.3497/) (2014)
- [Deep Reinforcement Learning Algorithm for Dynamic Pricing of Express Lanes with Multiple Access Locations](https://mlsanity.com/paper/1909.04760/) (2019)
- [Intelligent Task Offloading for Heterogeneous V2X Communications](https://mlsanity.com/paper/2006.15855/) (2020)
- [Approximation of Lorenz-Optimal Solutions in Multiobjective Markov Decision Processes](https://mlsanity.com/paper/1309.6856/) (2013)
- [Distributed Task Replication for Vehicular Edge Computing: Performance Analysis and Learning-based Algorithm](https://mlsanity.com/paper/2002.08833/) (2020)
- [Multi-Agent Distributed Reinforcement Learning for Making Decentralized Offloading Decisions](https://mlsanity.com/paper/2204.02267/) (2022)
- [Learning to Bid Long-Term: Multi-Agent Reinforcement Learning with Long-Term and Sparse Reward in Repeated Auction Games](https://mlsanity.com/paper/2204.02268/) (2022)

## Abstract (arXiv)

The Intelligent Transportation System (ITS) environment is known to be dynamic and distributed, where participants (vehicle users, operators, etc.) have multiple, changing and possibly conflicting objectives. Although Reinforcement Learning (RL) algorithms are commonly applied to optimize ITS applications such as resource management and offloading, most RL algorithms focus on single objectives. In many situations, converting a multi-objective problem into a single-objective one is impossible, intractable or insufficient, making such RL algorithms inapplicable. We propose a multi-objective, multi-agent reinforcement learning (MARL) algorithm with high learning efficiency and low computational requirements, which automatically triggers adaptive few-shot learning in a dynamic, distributed and noisy environment with sparse and delayed reward. We test our algorithm in an ITS environment with edge cloud computing. Empirical results show that the algorithm is quick to adapt to new environments and performs better in all individual and system metrics compared to the state-of-the-art benchmark. Our algorithm also addresses various practical concerns with its modularized and asynchronous online training method. In addition to the cloud simulation, we test our algorithm on a single-board computer and show that it can make inference in 6 milliseconds.