# Vision-Wireless Fusion for Multi-User Localization: A Cross-Modal Transformer Approach

arXiv: https://arxiv.org/abs/2609.23372 · HTML: https://mlsanity.com/paper/2609.23372/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 40 references that are in the map.
- First posted on arXiv 20 Sep 2026.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Millimeter Wave Base Stations with Cameras: Vision Aided Beam and Blockage Prediction](https://mlsanity.com/paper/1911.06255/) (2019)
- [Vision-Position Multi-Modal Beam Prediction Using Real Millimeter Wave Datasets](https://mlsanity.com/paper/2111.07574/) (2021)
- [ViWi: A Deep Learning Dataset Framework for Vision-Aided Wireless Communications](https://mlsanity.com/paper/1911.06257/) (2019)
- [5G MIMO Data for Machine Learning: Application to Beam-Selection using Deep Learning](https://mlsanity.com/paper/2106.05370/) (2021)
- [Learning to Localize: A 3D CNN Approach to User Positioning in Massive MIMO-OFDM Systems](https://mlsanity.com/paper/1910.12378/) (2019)
- [A Tutorial on 5G Positioning](https://mlsanity.com/paper/2311.10551/) (2023)
- [Multimodal-Wireless: A Large-Scale Dataset for Sensing and Communication](https://mlsanity.com/paper/2511.03220/) (2025)
- [Vision-Aided Radio: User Identity Match in Radio and Video Domains Using Machine Learning](https://mlsanity.com/paper/2010.07219/) (2020)
- [Multi-modal Image and Radio Frequency Fusion for Optimizing Vehicle Positioning](https://mlsanity.com/paper/2410.19788/) (2024)
- [Multi-modal Data based Semi-Supervised Learning for Vehicle Positioning](https://mlsanity.com/paper/2410.20680/) (2024)
- [Multimodal Radio and Vision Fusion for Robust Localization in Urban V2I Communications](https://mlsanity.com/paper/2508.17640/) (2025)
- [Vehicular Wireless Positioning -- A Survey](https://mlsanity.com/paper/2601.20547/) (2026)

## Abstract (arXiv)

Accurate multi-user localization is challenging in complex urban environments, where wireless measurements can become ambiguous under noise, blockage, and multipath, while visual observations provide complementary spatial context. This paper presents a vision-wireless fusion framework for multi-user localization using pilot-indexed channel state information (CSI). Orthogonal pilot indices preserve the identities of the communicating UEs in the CSI token sequence and localization outputs. The model encodes each pilot-indexed CSI observation as a query token and uses cross-attention to retrieve user-specific information from spatial visual memory. Self-attention among CSI tokens further captures inter-user interactions, while the resulting multimodal representations are used for user-wise localization. Experiments on different datasets show consistent improvements over model-based, CSI-only, and multimodal-fusion baselines. Further experiments evaluate the model under different wireless and visual conditions.