# Digital Twin Based Beam Prediction: Can we Train in the Digital World and Deploy in Reality?

arXiv: https://arxiv.org/abs/2301.07682 · HTML: https://mlsanity.com/paper/2301.07682/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jan 2023.

## Built on by (most cited first, 12 of 12)

- [Real-Time Digital Twins: Vision and Research Directions for 6G and Beyond](https://mlsanity.com/paper/2301.11283/) (2023)
- [Calibrating Wireless Ray Tracing for Digital Twinning using Local Phase Error Estimates](https://mlsanity.com/paper/2312.12625/) (2023)
- [Camera Based mmWave Beam Prediction: Towards Multi-Candidate Real-World Scenarios](https://mlsanity.com/paper/2308.06868/) (2023)
- [A Multi-Modal Simulation Framework to Enable Digital Twin-based V2X Communications in Dynamic Environments](https://mlsanity.com/paper/2303.06947/) (2023)
- [Vehicle Cameras Guide mmWave Beams: Approach and Real-World V2V Demonstration](https://mlsanity.com/paper/2308.10362/) (2023)
- [Automatic AI Model Selection for Wireless Systems: Online Learning via Digital Twinning](https://mlsanity.com/paper/2406.15819/) (2024)
- [Context-Aware Doubly-Robust Semi-Supervised Learning](https://mlsanity.com/paper/2502.15577/) (2025)
- [Digital Twin-Assisted Explainable AI for Robust Beam Prediction in mmWave MIMO Systems](https://mlsanity.com/paper/2507.14180/) (2025)
- [Vision-Assisted Digital Twin Creation for mmWave Beam Management](https://mlsanity.com/paper/2401.17781/) (2024)
- [Optimizing Reinforcement Learning Training over Digital Twin Enabled Multi-fidelity Networks](https://mlsanity.com/paper/2603.08931/) (2026)
- [TWIST: Closed-Loop token Synchronization for Application-Aware Wireless Digital Twins](https://mlsanity.com/paper/2605.27205/) (2026)
- [AIMS: An Agentic AI Framework for Sim-to-Real Multi-Modal ISAC](https://mlsanity.com/paper/2609.39964/) (2026)

## Abstract (arXiv)

Realizing the potential gains of large-scale MIMO systems requires the accurate estimation of their channels or the fine adjustment of their narrow beams. This, however, is typically associated with high channel acquisition/beam sweeping overhead that scales with the number of antennas. Machine and deep learning represent promising approaches to overcome these challenges thanks to their powerful ability to learn from prior observations and side information. Training machine and deep learning models, however, requires large-scale datasets that are expensive to collect in deployed systems. To address this challenge, we propose a novel direction that utilizes digital replicas of the physical world to reduce or even eliminate the MIMO channel acquisition overhead. In the proposed digital twin aided communication, 3D models that approximate the real-world communication environment are constructed and accurate ray-tracing is utilized to simulate the site-specific channels. These channels can then be used to aid various communication tasks. Further, we propose to use machine learning to approximate the digital replicas and reduce the ray tracing computational cost. To evaluate the proposed digital twin based approach, we conduct a case study focusing on the position-aided beam prediction task. The results show that a learning model trained solely with the data generated by the digital replica can achieve relatively good performance on the real-world data. Moreover, a small number of real-world data points can quickly achieve near-optimal performance, overcoming the modeling mismatches between the physical and digital worlds and significantly reducing the data acquisition overhead.