# Co-Optimizing Reconfigurable Environments and Policies for Decentralized Multi-Agent Navigation

arXiv: https://arxiv.org/abs/2403.14583 · HTML: https://mlsanity.com/paper/2403.14583/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 88 references that are in the map.
- First posted on arXiv 21 Mar 2024.

## Builds on (most cited first, 28 of 28)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [CVXPY: A Python-Embedded Modeling Language for Convex Optimization](https://mlsanity.com/paper/1603.00943/) (2016)
- [On Evaluation of Embodied Navigation Agents](https://mlsanity.com/paper/1807.06757/) (2018)
- [Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation](https://mlsanity.com/paper/1811.10092/) (2018)
- [Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning](https://mlsanity.com/paper/1805.01956/) (2018)
- [Navigating to Objects in the Real World](https://mlsanity.com/paper/2212.00922/) (2022)
- [Graph Neural Networks for Decentralized Multi-Robot Path Planning](https://mlsanity.com/paper/1912.06095/) (2019)
- [Learning Decentralized Controllers for Robot Swarms with Graph Neural Networks](https://mlsanity.com/paper/1903.10527/) (2019)
- [Jointly Learning to Construct and Control Agents using Deep Reinforcement Learning](https://mlsanity.com/paper/1801.01432/) (2018)
- [Scalable Co-Optimization of Morphology and Control in Embodied Machines](https://mlsanity.com/paper/1706.06133/) (2017)
- [Real-time optimal control via Deep Neural Networks: study on landing problems](https://mlsanity.com/paper/1610.08668/) (2016)
- [Prioritized Planning Algorithms for Trajectory Coordination of Multiple Mobile Robots](https://mlsanity.com/paper/1409.2399/) (2014)
- [Wide and Deep Graph Neural Network with Distributed Online Learning](https://mlsanity.com/paper/2107.09203/) (2021)
- [Adversarial Environment Generation for Learning to Navigate the Web](https://mlsanity.com/paper/2103.01991/) (2021)
- [Online Control Barrier Functions for Decentralized Multi-Agent Navigation](https://mlsanity.com/paper/2303.04313/) (2023)
- [Multi-Robot Path Deconfliction through Prioritization by Path Prospects](https://mlsanity.com/paper/1908.02361/) (2019)
- [Designing Environments Conducive to Interpretable Robot Behavior](https://mlsanity.com/paper/2007.00820/) (2020)
- [Learning Control Admissibility Models with Graph Neural Networks for Multi-Agent Navigation](https://mlsanity.com/paper/2210.09378/) (2022)
- [Environment Optimization for Multi-Agent Navigation](https://mlsanity.com/paper/2209.11279/) (2022)
- [Constrained Environment Optimization for Prioritized Multi-Agent Navigation](https://mlsanity.com/paper/2305.11260/) (2023)
- [SDP-based Joint Sensor and Controller Design for Information-regularized Optimal LQG Control](https://mlsanity.com/paper/1503.01848/) (2015)
- [Sensing-Constrained LQG Control](https://mlsanity.com/paper/1709.08826/) (2017)
- [Decentralized Motion Planning for Multi-Robot Navigation using Deep Reinforcement Learning](https://mlsanity.com/paper/2011.05605/) (2020)
- [Standby-Based Deadlock Avoidance Method for Multi-Agent Pickup and Delivery Tasks](https://mlsanity.com/paper/2201.06014/) (2022)
- [Escaping spurious local minimum trajectories in online time-varying nonconvex optimization](https://mlsanity.com/paper/1912.00561/) (2019)
- [Optimal WDM Power Allocation via Deep Learning for Radio on Free Space Optics Systems](https://mlsanity.com/paper/1906.09981/) (2019)

## Built on by (most cited first, 5 of 5)

- [Search-Aided Joint Agent-Environment Reinforcement Learning for Robust Lifelong Multi-Agent Path Finding with Rotations](https://mlsanity.com/paper/2608.05588/) (2026)
- [ReCoDe: Reinforcement Learning-based Dynamic Constraint Design for Multi-Agent Coordination](https://mlsanity.com/paper/2507.19151/) (2025)
- [Stochastic Sequential Decision Making over Expanding Networks with Graph Filtering](https://mlsanity.com/paper/2603.19501/) (2026)
- [GRASP: Gradient Realignment via Active Shared Perception for Multi-Agent Collaborative Optimization](https://mlsanity.com/paper/2604.00717/) (2026)
- [Distributed Multi Robot Lunar Cargo Transportation via Phase Decomposed Reinforcement Learning](https://mlsanity.com/paper/2607.00160/) (2026)

## Abstract (arXiv)

This work views the multi-agent system and its surrounding environment as a co-evolving system, where the behavior of one affects the other. The goal is to take both agent actions and environment configurations as decision variables, and optimize these two components in a coordinated manner to improve some measure of interest. Towards this end, we consider the problem of decentralized multi-agent navigation in a cluttered environment, where we assume that the layout of the environment is reconfigurable. By introducing two sub-objectives -- multi-agent navigation and environment optimization -- we propose an agent-environment co-optimization problem and develop a coordinated algorithm that alternates between these sub-objectives to search for an optimal synthesis of agent actions and environment configurations; ultimately, improving the navigation performance. Due to the challenge of explicitly modeling the relation between the agents, the environment and their performance therein, we leverage policy gradient to formulate a model-free learning mechanism within the coordinated framework. A formal convergence analysis shows that our coordinated algorithm tracks the local minimum solution of an associated time-varying non-convex optimization problem. Experiments corroborate theoretical findings and show the benefits of co-optimization. Interestingly, the results also indicate that optimized environments can offer structural guidance to de-conflict agents in motion.