mlsanityData to 7 Oct 2026

DT2: Decision-Targeted Digital Twins

arXiv 2606.25923 ↗First posted 24 Jun 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

A digital twin (DT) is a virtual model of a real-world system that can assist decision-making by simulating scenarios induced by different policies. However, typical machine learning-based DTs do not optimise for this use case. We prove that, when model capacity is limited, training DTs to minimise one-step transition errors can produce suboptimal models for ranking sets of policies according to a reward function. We further show that this holds empirically, even with expressive model classes. To address this, we introduce DT2, a decision-targeted DT training paradigm. Firstly, DT2 uses fitted Q-evaluation to estimate values of candidate policies from offline data. A DT is then trained to generate rollouts that preserve pairwise policy rankings derived from these proxy ground-truth values with an architecture-agnostic loss function. We empirically demonstrate the efficacy of our method across a range of settings and architectures. DT2 consistently improves policy ranking and reduces decision regret during policy selection relative to conventional DT training, both for policies used during training and for unseen policies, while maintaining a good level of raw simulation fidelity.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 38 of its 79 references that are in the map. First posted on arXiv 24 Jun 2026.

Builds on

38 of 79 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  6. 2018Neural Ordinary Differential Equationscited by 3,733
  7. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  8. 2019Augmented Neural ODEscited by 396
  9. 2015Doubly Robust Off-policy Value Evaluation for Reinforcement Learningcited by 337
  10. 2016Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learningcited by 285
  11. 2019DualDICE: Behavior-Agnostic Estimation of Discounted Stationary Distribution Correctionscited by 254
  12. 2019Batch Policy Learning under Constraintscited by 242
26 more
  1. 2018Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimationcited by 226
  2. 2018Melding the Data-Decisions Pipeline: Decision-Focused Learning for Combinatorial Optimizationcited by 220
  3. 2019Minimax Weight and Q-Function Learning for Off-Policy Evaluationcited by 150
  4. 2020GenDICE: Generalized Offline Estimation of Stationary Valuescited by 122
  5. 2019Empirical Study of Off-Policy Policy Evaluation for Reinforcement Learningcited by 121
  6. 2019Towards Optimal Off-Policy Evaluation for Reinforcement Learning with Marginalized Importance Samplingcited by 118
  7. 2021Benchmarks for Deep Off-Policy Evaluationcited by 89
  8. 2020The Value Equivalence Principle for Model-Based Reinforcement Learningcited by 88
  9. 2020GradientDICE: Rethinking Generalized Offline Estimation of Stationary Valuescited by 70
  10. 2021Physics-Integrated Variational Autoencoders for Robust and Interpretable Generative Modelingcited by 59
  11. 2017Consistent On-Line Off-Policy Evaluationcited by 51
  12. 2021Integrating Expert ODEs into Neural ODEs: Pharmacology and Disease Progressioncited by 48
  13. 2020POPCORN: Partially Observed Prediction COnstrained ReiNforcement Learningcited by 36
  14. 2021Control-Oriented Model-Based Reinforcement Learning with Implicit Differentiationcited by 35
  15. 2021Mismatched No More: Joint Model-Policy Optimization for Model-Based RLcited by 35
  16. 2022Value Gradient weighted Model-Based Reinforcement Learningcited by 31
  17. 2020DyNODE: Neural Ordinary Differential Equations for Dynamics Modeling in Continuous Controlcited by 17
  18. 2022Live in the Moment: Learning Dynamics Model Adapted to Evolving Policycited by 17
  19. 2024Automatically Learning Hybrid Digital Twins of Dynamical Systemscited by 13
  20. 2023TOM: Learning Policy-Aware Models for Model-Based Reinforcement Learning via Transition Occupancy Matchingcited by 10
  21. 2024Med-Real2Sim: Non-Invasive Medical Digital Twins using Physics-Informed Self-Supervised Learningcited by 8
  22. 2024DTR-Bench: An in silico Environment and Benchmark Platform for Reinforcement Learning Based Dynamic Treatment Regimecited by 8
  23. 2021Self-Consistent Models and Valuescited by 7
  24. 2022Decentralized digital twins of complex dynamical systemscited by 2
  25. 2026Model-based Offline RL via Robust Value-Aware Model Learning with Implicitly Differentiable Adaptive Weightingcited by 2
  26. 2023Decision-Focused Model-based Reinforcement Learning for Reward Transfercited by 1

Built on by

Not cited yet.