mlsanityData to 7 Oct 2026

Urban-Focused Multi-Task Offline Reinforcement Learning with Contrastive Data Sharing

arXiv 2406.14054 ↗First posted 20 Jun 2024cs.LGcited by 2

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

Enhancing diverse human decision-making processes in an urban environment is a critical issue across various applications, including ride-sharing vehicle dispatching, public transportation management, and autonomous driving. Offline reinforcement learning (RL) is a promising approach to learn and optimize human urban strategies (or policies) from pre-collected human-generated spatial-temporal urban data. However, standard offline RL faces two significant challenges: (1) data scarcity and data heterogeneity, and (2) distributional shift. In this paper, we introduce MODA -- a Multi-Task Offline Reinforcement Learning with Contrastive Data Sharing approach. MODA addresses the challenges of data scarcity and heterogeneity in a multi-task urban setting through Contrastive Data Sharing among tasks. This technique involves extracting latent representations of human behaviors by contrasting positive and negative data pairs. It then shares data presenting similar representations with the target task, facilitating data augmentation for each task. Moreover, MODA develops a novel model-based multi-task offline RL algorithm. This algorithm constructs a robust Markov Decision Process (MDP) by integrating a dynamics model with a Generative Adversarial Network (GAN). Once the robust MDP is established, any online RL or planning algorithm can be applied. Extensive experiments conducted in a real-world multi-task urban setting validate the effectiveness of MODA. The results demonstrate that MODA exhibits significant improvements compared to state-of-the-art baselines, showcasing its capability in advancing urban decision-making processes. We also made our code available to the research community.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 29 of its 48 references that are in the map. First posted on arXiv 20 Jun 2024.

Builds on

29 of 48 references.

Closest ancestors · 8

Same direction, newest first

  1. 2021Conservative Data Sharing for Multi-Task Offline Reinforcement Learning16 Sep
  2. 2021Offline Reinforcement Learning with Fisher Divergence Critic Regularization14 Mar
  3. 2020Offline Reinforcement Learning from Images with Latent Space Models21 Dec
  4. 2020Conservative Q-Learning for Offline Reinforcement Learning8 Jun
  5. 2020MOPO: Model-based Offline Policy Optimization27 May
  6. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems4 May
  7. 2020Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learning19 Feb
  8. 2019Behavior Regularized Offline Reinforcement Learning26 Nov

Other references · 21

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  5. 2016Generative Adversarial Imitation Learningcited by 1,576
  6. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  7. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  8. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  9. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  10. 2018Multi-task Deep Reinforcement Learning with PopArtcited by 138
  11. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  12. 2018Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendationcited by 101
9 more
  1. 2020Offline Meta-Reinforcement Learning with Advantage Weightingcited by 92
  2. 2020Rewriting History with Inverse RL: Hindsight Inference for Policy Improvementcited by 59
  3. 2021Offline Meta-Reinforcement Learning with Online Self-Supervisioncited by 52
  4. 2021Offline Reinforcement Learning for Autonomous Driving with Safety and Exploration Enhancementcited by 28
  5. 2022Offline Reinforcement Learning for Visual Navigationcited by 28
  6. 2019Competitive Experience Replaycited by 23
  7. 2019Policy Continuation with Hindsight Inverse Dynamicscited by 22
  8. 2019Reinforcement Learning without Ground-Truth Statecited by 15
  9. 2022Offline Reinforcement Learning for Road Traffic Controlcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Small Fleet, Big Impact: Enhancing Shared Micromobility Efficiency through Minimal Autonomous Vehicle Deploymentcited by 1
  2. 2026Bridging Policy and Real-World Dynamics: LLM-Augmented Rebalancing for Shared Micromobility Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Robust Task Representations for Offline Meta-Reinforcement Learning via Contrastive Learning
  2. 2024OffLight: An Offline Multi-Agent Reinforcement Learning Framework for Traffic Signal Control
  3. 2024Pessimistic Value Iteration for Multi-Task Data Sharing in Offline Reinforcement Learning
  4. 2024Offline Multi-task Transfer RL with Representational Penalization
  5. 2022Offline RL With Realistic Datasets: Heteroskedasticity and Support Constraints
  6. 2024Skills Regularized Task Decomposition for Multi-task Offline Reinforcement Learning
  7. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems
  8. 2021Representation Matters: Offline Pretraining for Sequential Decision Making