mlsanityData to 7 Oct 2026

Vision-Position Multi-Modal Beam Prediction Using Real Millimeter Wave Datasets

arXiv 2111.07574 ↗First posted 15 Nov 2021eess.SP cs.IT math.ITcited by 24

Not on the map. Why?

Abstract (arXiv)

Enabling highly-mobile millimeter wave (mmWave) and terahertz (THz) wireless communication applications requires overcoming the critical challenges associated with the large antenna arrays deployed at these systems. In particular, adjusting the narrow beams of these antenna arrays typically incurs high beam training overhead that scales with the number of antennas. To address these challenges, this paper proposes a multi-modal machine learning based approach that leverages positional and visual (camera) data collected from the wireless communication environment for fast beam prediction. The developed framework has been tested on a real-world vehicular dataset comprising practical GPS, camera, and mmWave beam training data. The results show the proposed approach achieves more than ≈ 75% top-1 beam prediction accuracy and close to 100% top-3 beam prediction accuracy in realistic communication scenarios.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Nov 2021.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 4; 2024: 3; 2025: 6; 2026: 9.

24 papers.

Sort:

By area:

  1. 2022DeepSense 6G: A Large-Scale Real-World Multi-Modal Sensing and Communication Datasetcited by 53
  2. 2023Multimodal Transformers for Wireless Communications: A Case Study in Beam Predictioncited by 8
  3. 2024Multi-Modal Transformer and Reinforcement Learning-based Beam Managementcited by 8
  4. 2023Camera Based mmWave Beam Prediction: Towards Multi-Candidate Real-World Scenarioscited by 6
  5. 2025Resource-Efficient Beam Prediction in mmWave Communications with Multimodal Realistic Simulation Frameworkcited by 5
  6. 2024DeepSense-V2V: A Vehicle-to-Vehicle Multi-Modal Sensing, Localization, and Communications Datasetcited by 4
  7. 2023Vehicle Cameras Guide mmWave Beams: Approach and Real-World V2V Demonstrationcited by 3
  8. 2024Generative AI Empowered LiDAR Point Cloud Generation with Multimodal Transformercited by 3
  9. 2025Beam Selection in ISAC using Contextual Bandit with Multi-modal Transformer and Transfer Learningcited by 3
  10. 2025Multi-Modality Sensing in mmWave Beamforming for Connected Vehicles Using Deep Learningcited by 3
  11. 2023Non-contact Multimodal Indoor Human Monitoring Systems: A Surveycited by 2
  12. 2026Communication-Efficient and Robust Multi-Modal Federated Learning via Latent-Space Consensuscited by 2
12 more
  1. 2022Millimeter Wave Drones with Cameras: Computer Vision Aided Wireless Beam Predictioncited by 1
  2. 2025Vision Aided Channel Prediction for Vehicular Communications: A Case Study of Received Power Prediction Using RGB Imagescited by 1
  3. 2026Equivariant Multi-agent Reinforcement Learning for Multimodal Vehicle-to-Infrastructure Systemscited by 1
  4. 2025GPS-Aided Deep Learning for Beam Prediction and Tracking in UAV mmWave Communicationcited by 0
  5. 2025Empower Low-Altitude Economy: A Reliability-Aware Dynamic Weighting Allocation for Multi-modal UAV Beam Predictioncited by 0
  6. 2026Agentic AI for Embodied-enhanced Beam Prediction in Low-Altitude Economy Networkscited by 0
  7. 2026Structure-Aware Multimodal LLM Framework for Trustworthy Near-Field Beam Predictioncited by 0
  8. 2026CSI-tuples-based 3D Channel Fingerprints Construction Assisted by MultiModal Learningcited by 0
  9. 2026Look Once, Beam Twice: Camera-Primed Real-Time Double-Directional mmWave Beam Management for Vehicular Connectivitycited by 0
  10. 2026Resolving Multi-Target Association in OFDM-based ISAC via Vision-aided Multi-Modal Learningcited by 0
  11. 2026CM-MAE: A Physics-Guided Cross-Modal Self-Supervised Learning Framework for Vision-Wireless Applicationscited by 0
  12. 2026Vision-Wireless Fusion for Multi-User Localization: A Cross-Modal Transformer Approachcited by 0