# A Bayesian Framework for Digital Twin-Based Control, Monitoring, and Data Collection in Wireless Systems

arXiv: https://arxiv.org/abs/2212.01351 · HTML: https://mlsanity.com/paper/2212.01351/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 74 references that are in the map.
- First posted on arXiv 2 Dec 2022.

## Builds on (most cited first, 30 of 39)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [Value-Decomposition Networks For Cooperative Multi-Agent Learning](https://mlsanity.com/paper/1706.05296/) (2017)
- [(More) Efficient Reinforcement Learning via Posterior Sampling](https://mlsanity.com/paper/1306.0940/) (2013)
- [Model-Based Deep Learning](https://mlsanity.com/paper/2012.08405/) (2020)
- [Model-Based Active Exploration](https://mlsanity.com/paper/1810.12162/) (2018)
- [Off-Policy Multi-Agent Decomposed Policy Gradients](https://mlsanity.com/paper/2007.12322/) (2020)
- [On Improving Deep Reinforcement Learning for POMDPs](https://mlsanity.com/paper/1804.06309/) (2018)
- [Digital-Twin-Enabled 6G: Vision, Architectural Trends, and Future Directions](https://mlsanity.com/paper/2102.12169/) (2021)
- [Bayesian Variational Autoencoders for Unsupervised Out-of-Distribution Detection](https://mlsanity.com/paper/1912.05651/) (2019)
- [A Comprehensive Review of Digital Twin -- Part 1: Modeling and Twinning Enabling Technologies](https://mlsanity.com/paper/2208.14197/) (2022)
- [Towards Joint Learning of Optimal MAC Signaling and Wireless Channel Access](https://mlsanity.com/paper/2007.09948/) (2020)
- [A Comprehensive Review of Digital Twin -- Part 2: Roles of Uncertainty Quantification and Optimization, a Battery Digital Twin, and Perspectives](https://mlsanity.com/paper/2208.12904/) (2022)
- [Deep Learning for Hybrid 5G Services in Mobile Edge Computing Systems: Learn from a Digital Twin](https://mlsanity.com/paper/1907.01523/) (2019)
- [Deep Reinforcement Learning for Stochastic Computation Offloading in Digital Twin Networks](https://mlsanity.com/paper/2011.08430/) (2020)
- [The Emergence of Wireless MAC Protocols with Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2108.07144/) (2021)
- [Edge Continual Learning for Dynamic Digital Twins over Wireless Networks](https://mlsanity.com/paper/2204.04795/) (2022)
- [Centralized Model and Exploration Policy for Multi-Agent RL](https://mlsanity.com/paper/2107.06434/) (2021)
- [Network Digital Twin: Context, Enabling Technologies and Opportunities](https://mlsanity.com/paper/2205.14206/) (2022)
- [Programmable and Customized Intelligence for Traffic Steering in 5G Networks Using Open RAN Architectures](https://mlsanity.com/paper/2209.14171/) (2022)
- [Towards a Decentralized Metaverse: Synchronized Orchestration of Digital Twins and Sub-Metaverses](https://mlsanity.com/paper/2211.14686/) (2022)
- [Digital Twin-Assisted Efficient Reinforcement Learning for Edge Task Scheduling](https://mlsanity.com/paper/2208.01781/) (2022)
- [Digital Twin for Networking: A Data-driven Performance Modeling Perspective](https://mlsanity.com/paper/2206.00310/) (2022)
- [The Paradigm of Digital Twin Communications](https://mlsanity.com/paper/2105.07182/) (2021)
- [Learning Generalized Wireless MAC Communication Protocols via Abstraction](https://mlsanity.com/paper/2206.06331/) (2022)
- [Dirichlet Bayesian Network Scores and the Maximum Relative Entropy Principle](https://mlsanity.com/paper/1708.00689/) (2017)
- [The Interplay of AI and Digital Twin: Bridging the Gap between Data-Driven and Model-Driven Approaches](https://mlsanity.com/paper/2209.12423/) (2022)

## Built on by (most cited first, 18 of 18)

- [Wireless Network Digital Twin for 6G: Generative AI as A Key Enabler](https://mlsanity.com/paper/2311.17451/) (2023)
- [Causal Reasoning: Charting a Revolutionary Course for Next-Generation AI-Native Wireless Networks](https://mlsanity.com/paper/2309.13223/) (2023)
- [Causal Semantic Communication for Digital Twins: A Generalizable Imitation Learning Approach](https://mlsanity.com/paper/2304.12502/) (2023)
- [Calibrating Wireless Ray Tracing for Digital Twinning using Local Phase Error Estimates](https://mlsanity.com/paper/2312.12625/) (2023)
- [In-Context Learning for MIMO Equalization Using Transformer-Based Sequence Models](https://mlsanity.com/paper/2311.06101/) (2023)
- [What If We Had Used a Different App? Reliable Counterfactual KPI Analysis in Wireless Systems](https://mlsanity.com/paper/2410.00150/) (2024)
- [How to Bridge the Sim-to-Real Gap in Digital Twin-Aided Telecommunication Networks](https://mlsanity.com/paper/2507.07067/) (2025)
- [Multi-Fidelity Bayesian Optimization With Across-Task Transferable Max-Value Entropy Search](https://mlsanity.com/paper/2403.09570/) (2024)
- [Automatic AI Model Selection for Wireless Systems: Online Learning via Digital Twinning](https://mlsanity.com/paper/2406.15819/) (2024)
- [Optimizing Wireless Resource Management and Synchronization in Digital Twin Networks](https://mlsanity.com/paper/2502.05116/) (2025)
- [Data-driven Energy Efficiency Modelling in Large-scale Networks: An Expert Knowledge and ML-based Approach](https://mlsanity.com/paper/2401.00443/) (2023)
- [Multi-Fidelity Hybrid Reinforcement Learning via Information Gain Maximization](https://mlsanity.com/paper/2509.14848/) (2025)
- [LLM-Based Agentic Negotiation for 6G: Addressing Uncertainty Neglect and Tail-Event Risk](https://mlsanity.com/paper/2511.19175/) (2025)
- [Should I Have Expressed a Different Intent? Counterfactual Generation for LLM-Based Autonomous Control](https://mlsanity.com/paper/2601.20090/) (2026)
- [Confounding-Valid Conformal Inference for Counterfactual KPIs in Wireless Networks](https://mlsanity.com/paper/2609.05073/) (2026)
- [Energy-Efficient Federated Learning and Migration in Digital Twin Edge Networks](https://mlsanity.com/paper/2503.15822/) (2025)
- [Should I Use This Synthetic Dataset for Training? How to Test with Minimal Real Data](https://mlsanity.com/paper/2608.27996/) (2026)
- [Conformal Risk-Averse Decision Making with Optimized Certainty Equivalent Risk Control](https://mlsanity.com/paper/2608.28179/) (2026)

## Abstract (arXiv)

Commonly adopted in the manufacturing and aerospace sectors, digital twin (DT) platforms are increasingly seen as a promising paradigm to control, monitor, and analyze software-based, "open", communication systems. Notably, DT platforms provide a sandbox in which to test artificial intelligence (AI) solutions for communication systems, potentially reducing the need to collect data and test algorithms in the field, i.e., on the physical twin (PT). A key challenge in the deployment of DT systems is to ensure that virtual control optimization, monitoring, and analysis at the DT are safe and reliable, avoiding incorrect decisions caused by "model exploitation". To address this challenge, this paper presents a general Bayesian framework with the aim of quantifying and accounting for model uncertainty at the DT that is caused by limitations in the amount and quality of data available at the DT from the PT. In the proposed framework, the DT builds a Bayesian model of the communication system, which is leveraged to enable core DT functionalities such as control via multi-agent reinforcement learning (MARL), monitoring of the PT for anomaly detection, prediction, data-collection optimization, and counterfactual analysis. To exemplify the application of the proposed framework, we specifically investigate a case-study system encompassing multiple sensing devices that report to a common receiver. Experimental results validate the effectiveness of the proposed Bayesian framework as compared to standard frequentist model-based solutions.