# Digital Twin-Enhanced Wireless Indoor Navigation: Achieving Efficient Environment Sensing with Zero-Shot Reinforcement Learning

arXiv: https://arxiv.org/abs/2306.06766 · HTML: https://mlsanity.com/paper/2306.06766/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 59 references that are in the map.
- First posted on arXiv 11 Jun 2023.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [On Evaluation of Embodied Navigation Agents](https://mlsanity.com/paper/1807.06757/) (2018)
- [Gibson Env: Real-World Perception for Embodied Agents](https://mlsanity.com/paper/1808.10654/) (2018)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [Semantic Scene Completion from a Single Depth Image](https://mlsanity.com/paper/1611.08974/) (2016)
- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [Learning to Explore using Active Neural SLAM](https://mlsanity.com/paper/2004.05155/) (2020)
- [Supersparse Linear Integer Models for Optimized Medical Scoring Systems](https://mlsanity.com/paper/1502.04269/) (2015)
- [Partially Observable Markov Decision Processes (POMDPs) and Robotics](https://mlsanity.com/paper/2107.07599/) (2021)
- [Digital Twin of Wireless Systems: Overview, Taxonomy, Challenges, and Opportunities](https://mlsanity.com/paper/2202.02559/) (2022)
- [Generalized Linear Rule Models](https://mlsanity.com/paper/1906.01761/) (2019)
- [The Confluence of Networks, Games and Learning](https://mlsanity.com/paper/2105.08158/) (2021)
- [Measuring Sample Efficiency and Generalization in Reinforcement Learning Benchmarks: NeurIPS 2020 Procgen Benchmark](https://mlsanity.com/paper/2103.15332/) (2021)
- [Physics-informed Dyna-Style Model-Based Deep Reinforcement Learning for Dynamic Control](https://mlsanity.com/paper/2108.00128/) (2021)
- [Millimeter Wave Cellular Wireless Networks: Potentials and Challenges](https://mlsanity.com/paper/1401.2560/) (2014)
- [On Convergence Rate of Adaptive Multiscale Value Function Approximation For Reinforcement Learning](https://mlsanity.com/paper/1908.08578/) (2019)
- [Blackwell Online Learning for Markov Decision Processes](https://mlsanity.com/paper/2012.14043/) (2020)
- [Scenario-Agnostic Zero-Trust Defense with Explainable Threshold Policy: A Meta-Learning Approach](https://mlsanity.com/paper/2303.03349/) (2023)
- [Causality and Batch Reinforcement Learning: Complementary Approaches To Planning In Unknown Domains](https://mlsanity.com/paper/2006.02579/) (2020)
- [From Physics-Based Models to Predictive Digital Twins via Interpretable Machine Learning](https://mlsanity.com/paper/2004.11356/) (2020)
- [Millimeter Wave Wireless Assisted Robot Navigation with Link State Classification](https://mlsanity.com/paper/2110.14789/) (2021)

## Built on by (most cited first, 11 of 11)

- [Automated Security Response through Online Learning with Adaptive Conjectures](https://mlsanity.com/paper/2402.12499/) (2024)
- [Symbiotic Game and Foundation Models for Cyber Deception Operations in Strategic Cyber Warfare](https://mlsanity.com/paper/2403.10570/) (2024)
- [From Texts to Shields: Convergence of Large Language Models and Cybersecurity](https://mlsanity.com/paper/2505.00841/) (2025)
- [Reinforcement Learning with Physics-Informed Symbolic Program Priors for Zero-Shot Wireless Indoor Navigation](https://mlsanity.com/paper/2506.22365/) (2025)
- [Stackelberg Coupling of Online Representation Learning and Reinforcement Learning](https://mlsanity.com/paper/2508.07452/) (2025)
- [Beyond Point Estimates: Likelihood-Based Full-Posterior Wireless Localization](https://mlsanity.com/paper/2509.25719/) (2025)
- [Transformer-Based Rate Prediction for Multi-Band Cellular Handsets](https://mlsanity.com/paper/2509.25722/) (2025)
- [Agentic Incident Response through Digital Twin-Enhanced Multiscale Planning](https://mlsanity.com/paper/2608.02422/) (2026)
- [When Simultaneous Localization and Mapping Meets Wireless Communications: A Survey](https://mlsanity.com/paper/2602.06995/) (2026)
- [LOCUS-DT: Localization via Observation-Conditioned Uncertainty Scoring with Digital Twins](https://mlsanity.com/paper/2608.00406/) (2026)
- [Hierarchical Agentic Incident Response with Digital-Twin-Validated Attack Inference](https://mlsanity.com/paper/2608.15016/) (2026)

## Abstract (arXiv)

Millimeter-wave (mmWave) communication is a vital component of future generations of mobile networks, offering not only high data rates but also precise beams, making it ideal for indoor navigation in complex environments. However, the challenges of multipath propagation and noisy signal measurements in indoor spaces complicate the use of mmWave signals for navigation tasks. Traditional physics-based methods, such as following the angle of arrival (AoA), often fall short in complex scenarios, highlighting the need for more sophisticated approaches. Digital twins, as virtual replicas of physical environments, offer a powerful tool for simulating and optimizing mmWave signal propagation in such settings. By creating detailed, physics-based models of real-world spaces, digital twins enable the training of machine learning algorithms in virtual environments, reducing the costs and limitations of physical testing. Despite their advantages, current machine learning models trained in digital twins often overfit specific virtual environments and require costly retraining when applied to new scenarios. In this paper, we propose a Physics-Informed Reinforcement Learning (PIRL) approach that leverages the physical insights provided by digital twins to shape the reinforcement learning (RL) reward function. By integrating physics-based metrics such as signal strength, AoA, and path reflections into the learning process, PIRL enables efficient learning and improved generalization to new environments without retraining. Our experiments demonstrate that the proposed PIRL, supported by digital twin simulations, outperforms traditional heuristics and standard RL models, achieving zero-shot generalization in unseen environments and offering a cost-effective, scalable solution for wireless indoor navigation.