# Multi-modal Data based Semi-Supervised Learning for Vehicle Positioning

arXiv: https://arxiv.org/abs/2410.20680 · HTML: https://mlsanity.com/paper/2410.20680/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 52 references that are in the map.
- First posted on arXiv 15 Oct 2024.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [Keystroke Dynamics as Part of Lifelogging](https://mlsanity.com/paper/2010.10144/) (2020)
- [Distributed Learning in Wireless Networks: Recent Progress and Future Challenges](https://mlsanity.com/paper/2104.02151/) (2021)
- [Convolutional Neural Networks at Constrained Time Cost](https://mlsanity.com/paper/1412.1710/) (2014)
- [Vision-Aided 6G Wireless Communications: Blockage Prediction and Proactive Handoff](https://mlsanity.com/paper/2102.09527/) (2021)
- [Millimeter Wave Base Stations with Cameras: Vision Aided Beam and Blockage Prediction](https://mlsanity.com/paper/1911.06255/) (2019)
- [Deep Learning on Multimodal Sensor Data at the Wireless Edge for Vehicular Network](https://mlsanity.com/paper/2201.04712/) (2022)
- [Self-Contrastive Learning based Semi-Supervised Radio Modulation Classification](https://mlsanity.com/paper/2203.15932/) (2022)
- [Fast Multi-Instance Multi-Label Learning](https://mlsanity.com/paper/1310.2049/) (2013)
- [Computer Vision Aided Beam Tracking in A Real-World Millimeter Wave Deployment](https://mlsanity.com/paper/2111.14803/) (2021)
- [An Equal-Size Hard EM Algorithm for Diverse Dialogue Generation](https://mlsanity.com/paper/2209.14627/) (2022)
- [Learning to Localize: A 3D CNN Approach to User Positioning in Massive MIMO-OFDM Systems](https://mlsanity.com/paper/1910.12378/) (2019)
- [Open World DETR: Transformer based Open World Object Detection](https://mlsanity.com/paper/2212.02969/) (2022)
- [ViWi Vision-Aided mmWave Beam Tracking: Dataset, Task, and Baseline Solutions](https://mlsanity.com/paper/2002.02445/) (2020)
- [Computer Vision Aided Blockage Prediction in Real-World Millimeter Wave Deployments](https://mlsanity.com/paper/2203.01907/) (2022)
- [Channel Estimation for Intelligent Reflecting Surface Assisted MIMO Systems: A Tensor Modeling Approach](https://mlsanity.com/paper/2008.04766/) (2020)
- [Positioning Using Wireless Networks: Applications, Recent Progress and Future Challenges](https://mlsanity.com/paper/2403.11417/) (2024)
- [CSI-Based Localization with CNNs Exploiting Phase Information](https://mlsanity.com/paper/2101.08983/) (2021)
- [CSI-fingerprinting Indoor Localization via Attention-Augmented Residual Convolutional Neural Network](https://mlsanity.com/paper/2205.05775/) (2022)
- [Reliable Deep Learning based Localization with CSI Fingerprints and Multiple Base Stations](https://mlsanity.com/paper/2111.11839/) (2021)
- [Multi-Environment based Meta-Learning with CSI Fingerprints for Radio Based Positioning](https://mlsanity.com/paper/2210.14510/) (2022)
- [Complex-Valued Neural Network based Federated Learning for Multi-user Indoor Positioning Performance Optimization](https://mlsanity.com/paper/2403.00665/) (2024)
- [Environment Sensing Considering the Occlusion Effect: A Multi-View Approach](https://mlsanity.com/paper/2207.00776/) (2022)
- [Positioning Using Visible Light Communications: A Perspective Arcs Approach](https://mlsanity.com/paper/2204.08131/) (2022)
- [Unsupervised Person Re-Identification with Wireless Positioning under Weak Scene Labeling](https://mlsanity.com/paper/2110.15610/) (2021)
- [Millimeter-Wave Downlink Positioning with a Single-Antenna Receiver](https://mlsanity.com/paper/1811.11586/) (2018)
- [Fingerprint Based mmWave Positioning System Aided by Reconfigurable Intelligent Surface](https://mlsanity.com/paper/2210.11998/) (2022)

## Built on by (most cited first, 2 of 2)

- [Equivariant Multi-agent Reinforcement Learning for Multimodal Vehicle-to-Infrastructure Systems](https://mlsanity.com/paper/2604.06914/) (2026)
- [Vision-Wireless Fusion for Multi-User Localization: A Cross-Modal Transformer Approach](https://mlsanity.com/paper/2609.23372/) (2026)

## Abstract (arXiv)

In this paper, a multi-modal data based semi-supervised learning (SSL) framework that jointly use channel state information (CSI) data and RGB images for vehicle positioning is designed. In particular, an outdoor positioning system where the vehicle locations are determined by a base station (BS) is considered. The BS equipped with several cameras can collect a large amount of unlabeled CSI data and a small number of labeled CSI data of vehicles, and the images taken by cameras. Although the collected images contain partial information of vehicles (i.e. azimuth angles of vehicles), the relationship between the unlabeled CSI data and its azimuth angle, and the distances between the BS and the vehicles captured by images are both unknown. Therefore, the images cannot be directly used as the labels of unlabeled CSI data to train a positioning model. To exploit unlabeled CSI data and images, a SSL framework that consists of a pretraining stage and a downstream training stage is proposed. In the pretraining stage, the azimuth angles obtained from the images are considered as the labels of unlabeled CSI data to pretrain the positioning model. In the downstream training stage, a small sized labeled dataset in which the accurate vehicle positions are considered as labels is used to retrain the model. Simulation results show that the proposed method can reduce the positioning error by up to 30% compared to a baseline where the model is not pretrained.