# Multi-Modality Sensing in mmWave Beamforming for Connected Vehicles Using Deep Learning

arXiv: https://arxiv.org/abs/2504.06173 · HTML: https://mlsanity.com/paper/2504.06173/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 71 references that are in the map.
- First posted on arXiv 8 Apr 2025.

## Builds on (most cited first, 30 of 39)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [DeepSense 6G: A Large-Scale Real-World Multi-Modal Sensing and Communication Dataset](https://mlsanity.com/paper/2211.09769/) (2022)
- [Vision-Position Multi-Modal Beam Prediction Using Real Millimeter Wave Datasets](https://mlsanity.com/paper/2111.07574/) (2021)
- [LiDAR Aided Future Beam Prediction in Real-World Millimeter Wave V2I Communications](https://mlsanity.com/paper/2203.05548/) (2022)
- [A Survey of Beam Management for mmWave and THz Communications Towards 6G](https://mlsanity.com/paper/2308.02135/) (2023)
- [LIDAR Data for Deep Learning-Based mmWave Beam-Selection](https://mlsanity.com/paper/1908.07488/) (2019)
- [Deep Learning on Multimodal Sensor Data at the Wireless Edge for Vehicular Network](https://mlsanity.com/paper/2201.04712/) (2022)
- [Computer Vision Aided mmWave Beam Alignment in V2X Communications](https://mlsanity.com/paper/2207.11409/) (2022)
- [Position Aided Beam Prediction in the Real World: How Useful GPS Locations Actually Are?](https://mlsanity.com/paper/2205.09054/) (2022)
- [Federated mmWave Beam Selection Utilizing LIDAR Data](https://mlsanity.com/paper/2102.02802/) (2021)
- [AI/ML for Beam Management in 5G-Advanced: A Standardization Perspective](https://mlsanity.com/paper/2309.10575/) (2023)
- [LIDAR and Position-Aided mmWave Beam Selection with Non-local CNNs and Curriculum Training](https://mlsanity.com/paper/2104.14579/) (2021)
- [Deep Learning-based Link Configuration for Radar-aided Multiuser mmWave Vehicle-to-Infrastructure Communication](https://mlsanity.com/paper/2201.04657/) (2022)
- [Advancing Ultra-Reliable 6G: Transformer and Semantic Localization Empowered Robust Beamforming in Millimeter-Wave Communications](https://mlsanity.com/paper/2406.02000/) (2024)
- [Proactively Predicting Dynamic 6G Link Blockages Using LiDAR and In-Band Signatures](https://mlsanity.com/paper/2211.09535/) (2022)
- [Multiverse at the Edge: Interacting Real World and Digital Twins for Wireless Beamforming](https://mlsanity.com/paper/2305.10350/) (2023)
- [Environment Semantic Communication: Enabling Distributed Sensing Aided Networks](https://mlsanity.com/paper/2402.14766/) (2024)
- [A Deep Learning Approach to Location- and Orientation-aided 3D Beam Selection for mmWave Communications](https://mlsanity.com/paper/2110.06859/) (2021)
- [Point Cloud-based Proactive Link Quality Prediction for Millimeter-wave Communications](https://mlsanity.com/paper/2301.00752/) (2023)
- [mmWave Enabled Connected Autonomous Vehicles: A Use Case with V2V Cooperative Perception](https://mlsanity.com/paper/2310.09700/) (2023)
- [Sub-6GHz Assisted mmWave Hybrid Beamforming with Heterogeneous Graph Neural Network](https://mlsanity.com/paper/2404.17138/) (2024)
- [DeepSense-V2V: A Vehicle-to-Vehicle Multi-Modal Sensing, Localization, and Communications Dataset](https://mlsanity.com/paper/2406.17908/) (2024)
- [Location-Aided Beamforming in Mobile Millimeter-Wave Networks](https://mlsanity.com/paper/2205.09887/) (2022)
- [Vision Aided Environment Semantics Extraction and Its Application in mmWave Beam Selection](https://mlsanity.com/paper/2301.08973/) (2023)
- [Environment Semantic Aided Communication: A Real World Demonstration for Beam Prediction](https://mlsanity.com/paper/2302.06736/) (2023)
- [Millimeter Wave V2V Beam Tracking using Radar: Algorithms and Real-World Demonstration](https://mlsanity.com/paper/2308.01558/) (2023)
- [Vehicle Cameras Guide mmWave Beams: Approach and Real-World V2V Demonstration](https://mlsanity.com/paper/2308.10362/) (2023)
- [Position Aware 60 GHz mmWave Beamforming for V2V Communications Utilizing Deep Learning](https://mlsanity.com/paper/2402.01259/) (2024)

## Built on by (most cited first, 3 of 3)

- [Sheaf-Based Decentralized Multimodal Learning for Next-Generation Wireless Communication Systems](https://mlsanity.com/paper/2506.22374/) (2025)
- [QuaMoE-DRF: Proactive Beam and Rate Adaptation via Multimodal Dynamic Radio Map Forecasting in ISAC Networks](https://mlsanity.com/paper/2607.00974/) (2026)
- [Multi-Modal Sensing and Fusion in mmWave Beamforming for Connected Vehicles: A Transformer Based Framework](https://mlsanity.com/paper/2602.13606/) (2026)

## Abstract (arXiv)

Beamforming techniques are considered as essential parts to compensate severe path losses in millimeter-wave (mmWave) communications. In particular, these techniques adopt large antenna arrays and formulate narrow beams to obtain satisfactory received powers. However, performing accurate beam alignment over narrow beams for efficient link configuration by traditional standard defined beam selection approaches, which mainly rely on channel state information and beam sweeping through exhaustive searching, imposes computational and communications overheads. And, such resulting overheads limit their potential use in vehicle-to-infrastructure (V2I) and vehicle-to-vehicle (V2V) communications involving highly dynamic scenarios. In comparison, utilizing out-of-band contextual information, such as sensing data obtained from sensor devices, provides a better alternative to reduce overheads. This paper presents a deep learning-based solution for utilizing the multi-modality sensing data for predicting the optimal beams having sufficient mmWave received powers so that the best V2I and V2V line-of-sight links can be ensured proactively. The proposed solution has been tested on real-world measured mmWave sensing and communication data, and the results show that it can achieve up to 98.19% accuracies while predicting top-13 beams. Correspondingly, when compared to existing been sweeping approach, the beam sweeping searching space and time overheads are greatly shortened roughly by 79.67% and 91.89%, respectively which confirm a promising solution for beamforming in mmWave enabled communications.