# Can Optimal Transport Improve Federated Inverse Reinforcement Learning?

arXiv: https://arxiv.org/abs/2601.00309 · HTML: https://mlsanity.com/paper/2601.00309/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 13 of its 25 references that are in the map.
- First posted on arXiv 1 Jan 2026.

## Placed in

- Offline RL and imitation learning from data / [Inverse reinforcement learning](https://mlsanity.com/offlinerl/inverse-reinforcement-learning/)

## Builds on (most cited first, 13 of 13)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Federated Reinforcement Learning: Techniques, Applications, and Open Challenges](https://mlsanity.com/paper/2108.11887/) (2021)
- [Multi-task Maximum Entropy Inverse Reinforcement Learning](https://mlsanity.com/paper/1805.08882/) (2018)
- [FedAR: Activity and Resource-Aware Federated Learning Model for Distributed Mobile Robots](https://mlsanity.com/paper/2101.03705/) (2021)
- [Tiny Robot Learning: Challenges and Directions for Machine Learning in Resource-Constrained Robots](https://mlsanity.com/paper/2205.05748/) (2022)
- [Revisiting Maximum Entropy Inverse Reinforcement Learning: New Perspectives and Algorithms](https://mlsanity.com/paper/2012.00889/) (2020)
- [Understanding Reward Ambiguity Through Optimal Transport Theory in Inverse Reinforcement Learning](https://mlsanity.com/paper/2310.12055/) (2023)
- [Heterogeneous Federated Reinforcement Learning Using Wasserstein Barycenters](https://mlsanity.com/paper/2506.15825/) (2025)
- [Enhancing Spectrum Efficiency in 6G Satellite Networks: A GAIL-Powered Policy Learning via Asynchronous Federated Inverse Reinforcement Learning](https://mlsanity.com/paper/2409.18718/) (2024)
- [Wasserstein-Barycenter Consensus for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2506.12497/) (2025)
- [Wasserstein Barycenter Soft Actor-Critic](https://mlsanity.com/paper/2506.10167/) (2025)

## Built on by (most cited first, 1 of 1)

- [Revisiting On-policy Adversarial Black-Box Distillation: Calibrating Groupwise Reward Geometry for Effective Advantage Construction](https://mlsanity.com/paper/2609.39757/) (2026)

## Abstract (arXiv)

In robotics and multi-agent systems, fleets of autonomous agents often operate in subtly different environments while pursuing a common high-level objective. Directly pooling their data to learn a shared reward function is typically impractical due to differences in dynamics, privacy constraints, and limited communication bandwidth. This paper introduces an optimal transport-based approach to federated inverse reinforcement learning (IRL). Each client first performs lightweight Maximum Entropy IRL locally, adhering to its computational and privacy limitations. The resulting reward functions are then fused via a Wasserstein barycenter, which considers their underlying geometric structure. We further prove that this barycentric fusion yields a more faithful global reward estimate than conventional parameter averaging methods in federated learning. Overall, this work provides a principled and communication-efficient framework for deriving a shared reward that generalizes across heterogeneous agents and environments.