# Vision-Assisted Digital Twin Creation for mmWave Beam Management

arXiv: https://arxiv.org/abs/2401.17781 · HTML: https://mlsanity.com/paper/2401.17781/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 15 references that are in the map.
- First posted on arXiv 31 Jan 2024.

## Builds on (most cited first, 10 of 10)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction](https://mlsanity.com/paper/2206.00665/) (2022)
- [Attention Attention Everywhere: Monocular Depth Prediction with Skip Attention](https://mlsanity.com/paper/2210.09071/) (2022)
- [Sionna RT: Differentiable Ray Tracing for Radio Propagation Modeling](https://mlsanity.com/paper/2303.11103/) (2023)
- [DeepSense 6G: A Large-Scale Real-World Multi-Modal Sensing and Communication Dataset](https://mlsanity.com/paper/2211.09769/) (2022)
- [Millimeter Wave Base Stations with Cameras: Vision Aided Beam and Blockage Prediction](https://mlsanity.com/paper/1911.06255/) (2019)
- [Digital Twin Based Beam Prediction: Can we Train in the Digital World and Deploy in Reality?](https://mlsanity.com/paper/2301.07682/) (2023)
- [WiThRay: A Versatile Ray-Tracing Simulator for Smart Wireless Environments](https://mlsanity.com/paper/2304.11385/) (2023)
- [A Digital Twin Assisted Framework for Interference Nulling in Millimeter Wave MIMO Systems](https://mlsanity.com/paper/2301.13311/) (2023)

## Built on by (most cited first, 1 of 1)

- [Look Once, Beam Twice: Camera-Primed Real-Time Double-Directional mmWave Beam Management for Vehicular Connectivity](https://mlsanity.com/paper/2605.05071/) (2026)

## Abstract (arXiv)

In the context of communication networks, digital twin technology provides a means to replicate the radio frequency (RF) propagation environment as well as the system behaviour, allowing for a way to optimize the performance of a deployed system based on simulations. One of the key challenges in the application of Digital Twin technology to mmWave systems is the prevalent channel simulators' stringent requirements on the accuracy of the 3D Digital Twin, reducing the feasibility of the technology in real applications. We propose a practical Digital Twin creation pipeline and a channel simulator, that relies only on a single mounted camera and position information. We demonstrate the performance benefits compared to methods that do not explicitly model the 3D environment, on downstream sub-tasks in beam acquisition, using the real-world dataset of the DeepSense6G challenge