mlsanityData to 14 Oct 2026

TwinLoop: Simulation-in-the-Loop Digital Twins for Online Multi-Agent Reinforcement Learning

arXiv 2604.06610 ↗First posted 8 Apr 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Decentralised online learning enables runtime adaptation in cyber-physical multi-agent systems, but when operating conditions change, learned policies often require substantial trial-and-error interaction before recovering performance. To address this, we propose TwinLoop, a simulation-in-the-loop digital twin framework for online multi-agent reinforcement learning. When a context shift occurs, the digital twin is triggered to reconstruct the current system state, initialise from the latest agent policies, and perform accelerated policy improvement with simulation what-if analysis before synchronising updated parameters back to the agents in the physical system. We evaluate TwinLoop in a vehicular edge computing task-offloading scenario with changing workload and infrastructure conditions. The results suggest that digital twins can improve post-shift adaptation efficiency and reduce reliance on costly online trial-and-error.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 28 references that are in the map. First posted on arXiv 8 Apr 2026.

Builds on

14 of 28 references.

References · 14

  1. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  2. 2020A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environmentscited by 62
  3. 2020Adaptive Federated Learning and Digital Twin for Industrial Internet of Thingscited by 27
  4. 2022Digital Twin-Assisted Efficient Reinforcement Learning for Edge Task Schedulingcited by 11
  5. 2019Feature-Model-Guided Online Learning for Self-Adaptive Systemscited by 7
  6. 2021Applying Machine Learning in Self-Adaptive Systems: A Systematic Literature Reviewcited by 7
  7. 2021Decentralized Self-Adaptive Systems: A Mapping Studycited by 5
  8. 2024Large Language Models for Explainable Decisions in Dynamic Digital Twinscited by 4
  9. 2022Knowledge Equivalence in Digital Twins of Intelligent Systemscited by 3
  10. 2022Explainable Human-in-the-loop Dynamic Data-Driven Digital Twinscited by 3
  11. 2024Digital Twin Vehicular Edge Computing Network: Task Offloading and Resource Allocationcited by 3
  12. 2023Dynamic Data-Driven Digital Twins for Blockchain Systemscited by 2
2 more
  1. 2025Intelligent Offloading in Vehicular Edge Computing: A Comprehensive Review of Deep Reinforcement Learning Approaches and Architecturescited by 1
  2. 2025A Digital Twin-based Multi-Agent Reinforcement Learning Framework for Vehicle-to-Grid Coordinationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Mixed-Reality Digital Twins: Leveraging the Physical and Virtual Worlds for Hybrid Sim2Real Transition of Multi-Agent Reinforcement Learning Policies
  2. 2024Digital Twin-Driven Reinforcement Learning for Obstacle Avoidance in Robot Manipulators: A Self-Improving Online Training Framework
  3. 2024Toward Enhanced Reinforcement Learning-Based Resource Management via Digital Twin: Opportunities, Applications, and Challenges
  4. 2026Sim2O: Efficient Offline-to-Online MARL via Joint Action Composition
  5. 2024A Digital Twin Framework for Reinforcement Learning with Real-Time Self-Improvement via Human Assistive Teleoperation
  6. 2025Toward Trustworthy Digital Twins in AI Agent-based Wireless Network Optimization: Challenges, Solutions, and Opportunities
  7. 2023Digital Twin-Enhanced Deep Reinforcement Learning for Resource Management in Networks Slicing
  8. 2026Leveraging Human-In-The-Loop Demonstrations in Reinforcement Learning for Digital Twin-Driven Robot Flexibility