# Resolving Multi-Target Association in OFDM-based ISAC via Vision-aided Multi-Modal Learning

arXiv: https://arxiv.org/abs/2606.22195 · HTML: https://mlsanity.com/paper/2606.22195/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 40 references that are in the map.
- First posted on arXiv 20 Jun 2026.

## Builds on (most cited first, 24 of 24)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Vision-Aided 6G Wireless Communications: Blockage Prediction and Proactive Handoff](https://mlsanity.com/paper/2102.09527/) (2021)
- [Vision-Position Multi-Modal Beam Prediction Using Real Millimeter Wave Datasets](https://mlsanity.com/paper/2111.07574/) (2021)
- [A Survey on Fundamental Limits of Integrated Sensing and Communication](https://mlsanity.com/paper/2104.09954/) (2021)
- [Computer Vision Aided mmWave Beam Alignment in V2X Communications](https://mlsanity.com/paper/2207.11409/) (2022)
- [Computer Vision Aided Beam Tracking in A Real-World Millimeter Wave Deployment](https://mlsanity.com/paper/2111.14803/) (2021)
- [Multi-Modal Beam Prediction Challenge 2022: Towards Generalization](https://mlsanity.com/paper/2209.07519/) (2022)
- [The Integrated Sensing and Communication Revolution for 6G: Vision, Techniques, and Applications](https://mlsanity.com/paper/2405.01816/) (2024)
- [LIDAR and Position-Aided mmWave Beam Selection with Non-local CNNs and Curriculum Training](https://mlsanity.com/paper/2104.14579/) (2021)
- [Fundamental CRB-Rate Tradeoff in Multi-Antenna ISAC Systems with Information Multicasting and Multi-Target Sensing](https://mlsanity.com/paper/2307.11337/) (2023)
- [Multimodal Deep Learning-Empowered Beam Prediction in Future THz ISAC Systems](https://mlsanity.com/paper/2505.02381/) (2025)
- [Vision Aided Environment Semantics Extraction and Its Application in mmWave Beam Selection](https://mlsanity.com/paper/2301.08973/) (2023)
- [ISAC-NET: Model-driven Deep Learning for Integrated Passive Sensing and Communication](https://mlsanity.com/paper/2307.15074/) (2023)
- [Joint Sensing and Task-Oriented Communications with Image and Wireless Data Modalities for Dynamic Spectrum Access](https://mlsanity.com/paper/2312.13931/) (2023)
- [Deep Learning Based Multi-Node ISAC 4D Environmental Reconstruction with Uplink- Downlink Cooperation](https://mlsanity.com/paper/2404.14862/) (2024)
- [Deep Learning-based Techniques for Integrated Sensing and Communication Systems: State-of-the-Art, Challenges, and Opportunities](https://mlsanity.com/paper/2509.06968/) (2025)
- [Joint Active and Passive Beamforming Design for IRS-Aided Radar-Communication](https://mlsanity.com/paper/2203.14532/) (2022)
- [Integrated Sensing and Communication: Joint Pilot and Transmission Design](https://mlsanity.com/paper/2211.12891/) (2022)
- [Sensing Aided Uplink Transmission in OTFS ISAC with Joint Parameter Association, Channel Estimation and Signal Detection](https://mlsanity.com/paper/2305.11548/) (2023)
- [3D Multi-Target Localization Via Intelligent Reflecting Surface: Protocol and Analysis](https://mlsanity.com/paper/2310.15574/) (2023)
- [MIMO-OFDM ISAC Waveform Design for Range-Doppler Sidelobe Suppression](https://mlsanity.com/paper/2406.17218/) (2024)
- [Network-level ISAC: An Analytical Study of Antenna Topologies Ranging from Massive to Cell-Free MIMO](https://mlsanity.com/paper/2410.06365/) (2024)
- [Low-Complexity Minimum BER Precoder Design for ISAC Systems: A Delay-Doppler Perspective](https://mlsanity.com/paper/2410.15646/) (2024)
- [Vision-Aided ISAC in Low-Altitude Economy Networks via De-Diffused Visual Priors](https://mlsanity.com/paper/2507.01574/) (2025)

## Abstract (arXiv)

Orthogonal frequency division multiplexing (OFDM)-based integrated sensing and communication (ISAC) systems commonly extract target parameters by peak-searching a delay-Doppler map (DDM) constructed from reflected pilots. In multi-target scenarios, this results in ambiguity: the DDM does not reveal which physical target produced which peak, and two targets within the same delay-Doppler resolution cell cannot be separated. We propose a vision-assisted OFDM-ISAC framework that resolves both limitations by fusing wireless and visual modalities. The transmitter encodes an onboard street-view image with deep joint source-channel coding (DeepJSCC) and transmits it over the same OFDM waveform used for sensing; the receiver reconstructs the image, runs a fine-tuned YOLOv5 detector and fuses the resulting per-target features (bounding-box coordinates and class labels) with the DDM and transmitter-receiver geometry through a learned multi-modal network. To stabilize training of the high dimensional delay and Doppler classifiers, we introduce a Kullback Leibler loss against triangular soft labels centered on the ground-truth bin. On a Blender-rendered vehicular testbed, the proposed framework achieves a 16 cm localization root mean square error (RMSE) and a 10.8 ns delay RMSE. An ablation study confirms that removing the visual modality causes a 60x degradation in localization. These results highlight the potential of vision to overcome the data-association and resolution limits of single-modality ISAC.