# Look Once, Beam Twice: Camera-Primed Real-Time Double-Directional mmWave Beam Management for Vehicular Connectivity

arXiv: https://arxiv.org/abs/2605.05071 · HTML: https://mlsanity.com/paper/2605.05071/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 45 references that are in the map.
- First posted on arXiv 6 May 2026.

## Builds on (most cited first, 16 of 16)

- [YOLOv11: An Overview of the Key Architectural Enhancements](https://mlsanity.com/paper/2410.17725/) (2024)
- [DeepSense 6G: A Large-Scale Real-World Multi-Modal Sensing and Communication Dataset](https://mlsanity.com/paper/2211.09769/) (2022)
- [Vision-Aided 6G Wireless Communications: Blockage Prediction and Proactive Handoff](https://mlsanity.com/paper/2102.09527/) (2021)
- [Millimeter Wave Base Stations with Cameras: Vision Aided Beam and Blockage Prediction](https://mlsanity.com/paper/1911.06255/) (2019)
- [Vision-Position Multi-Modal Beam Prediction Using Real Millimeter Wave Datasets](https://mlsanity.com/paper/2111.07574/) (2021)
- [LiDAR Aided Future Beam Prediction in Real-World Millimeter Wave V2I Communications](https://mlsanity.com/paper/2203.05548/) (2022)
- [A Survey of Beam Management for mmWave and THz Communications Towards 6G](https://mlsanity.com/paper/2308.02135/) (2023)
- [Radar Aided 6G Beam Prediction: Deep Learning Algorithms and Real-World Demonstration](https://mlsanity.com/paper/2111.09676/) (2021)
- [Computer Vision Aided Beam Tracking in A Real-World Millimeter Wave Deployment](https://mlsanity.com/paper/2111.14803/) (2021)
- [Camera Based mmWave Beam Prediction: Towards Multi-Candidate Real-World Scenarios](https://mlsanity.com/paper/2308.06868/) (2023)
- [Machine Learning on Camera Images for Fast mmWave Beamforming](https://mlsanity.com/paper/2102.07337/) (2021)
- [Environment Semantic Aided Communication: A Real World Demonstration for Beam Prediction](https://mlsanity.com/paper/2302.06736/) (2023)
- [A Tale of Two Mobile Generations: 5G-Advanced and 6G in 3GPP Release 20](https://mlsanity.com/paper/2506.11828/) (2025)
- [Millimeter-Wave V2X Channels: PropagationStatistics, Beamforming, and Blockage](https://mlsanity.com/paper/1808.01001/) (2018)
- [Vision-Assisted Digital Twin Creation for mmWave Beam Management](https://mlsanity.com/paper/2401.17781/) (2024)
- [Machine Learning-Based mmWave MIMO Beam Tracking in V2I Scenarios: Algorithms and Datasets](https://mlsanity.com/paper/2412.05427/) (2024)

## Abstract (arXiv)

Millimeter-wave (mmWave) frequencies promise multi-gigabit connectivity for vehicle-to-everything (V2X) networks, but face challenges in terms of severe path loss and mobility-related beam misalignment. Reliable V2X connectivity requires fast, double-directional beam alignment. However, existing methods suffer from high training overhead and limited generalization to unseen scenarios. This paper presents VIsion-based BEamforming(VIBE), a hybrid model-based, closed-loop, learning architecture for real-time double-directional mmWave beam management primed by camera sensing. VIBE fuses machine learning, model-based reasoning, and closed-loop RF feedback to balance beam-pair establishment latency with link quality. VIBE bypasses exhaustive training overhead and accelerates link establishment by leveraging camera observations to reduce the beam-search space. Lightweight beam refinement and offset tracking mechanisms adaptively refine beams in response to dynamic application requirements. VIBE is implemented and evaluated across online indoor/outdoor testbeds, public datasets, and real-time vehicular experiments, demonstrating strong generalization capabilities, making it suitable for real-time V2X communication. Comparisons with 5G NR hierarchical beamforming show that VIBE consistently maintains lower outage rates. Furthermore, VIBE outperforms state-of-the-art end-to-end ML models for beam selection when evaluated on public datasets and achieves outage rates as low as 1.1-1.4 %. The results show that a hybrid model-based, closed-loop learning architecture is better suited for real-world mmWave vehicular connectivity than end-to-end trained ML models. For reproducibility, we publish our code to https://github.com/UNL-CPN-Lab/Look-Once-Beam-Twice.