# AIMS: An Agentic AI Framework for Sim-to-Real Multi-Modal ISAC

arXiv: https://arxiv.org/abs/2609.39964 · HTML: https://mlsanity.com/paper/2609.39964/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Agentic AI for wireless networks, one of 16 directions in LLM agents and tool use.
- Builds on 23 of its 30 references that are in the map.
- First posted on arXiv 30 Sep 2026.

## Placed in

- LLM agents and tool use / [Agentic AI for wireless networks](https://mlsanity.com/agents/agentic-ai-for-wireless-networks/)

## Builds on (most cited first, 23 of 23)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [The Roadmap to 6G -- AI Empowered Wireless Networks](https://mlsanity.com/paper/1904.11686/) (2019)
- [Sionna: An Open-Source Library for Next-Generation Physical Layer Research](https://mlsanity.com/paper/2203.11854/) (2022)
- [DeepMIMO: A Generic Deep Learning Dataset for Millimeter Wave and Massive MIMO Applications](https://mlsanity.com/paper/1902.06435/) (2019)
- [Integrated Sensing and Communications: Towards Dual-functional Wireless Networks for 6G and Beyond](https://mlsanity.com/paper/2108.07165/) (2021)
- [DeepSense 6G: A Large-Scale Real-World Multi-Modal Sensing and Communication Dataset](https://mlsanity.com/paper/2211.09769/) (2022)
- [WirelessLLM: Empowering Large Language Models Towards Wireless Intelligence](https://mlsanity.com/paper/2405.17053/) (2024)
- [WirelessAgent: Large Language Model Agents for Intelligent Wireless Networks](https://mlsanity.com/paper/2505.01074/) (2025)
- [Digital Twin Based Beam Prediction: Can we Train in the Digital World and Deploy in Reality?](https://mlsanity.com/paper/2301.07682/) (2023)
- [Towards Intelligent Millimeter and Terahertz Communication for 6G: Computer Vision-aided Beamforming](https://mlsanity.com/paper/2209.02258/) (2022)
- [ComAgent: Multi-LLM based Agentic AI Empowered Intelligent Wireless Networks](https://mlsanity.com/paper/2601.19607/) (2026)
- [Deep learning for location based beamforming with NLOS channels](https://mlsanity.com/paper/2201.01386/) (2021)
- [Environment Semantics Aided Wireless Communications: A Case Study of mmWave Beam Prediction and Blockage Prediction](https://mlsanity.com/paper/2301.05837/) (2023)
- [WirelessAgent++: Automated Agentic Workflow Design and Benchmarking for Wireless Networks](https://mlsanity.com/paper/2603.00501/) (2026)
- [Multimodal-Wireless: A Large-Scale Dataset for Sensing and Communication](https://mlsanity.com/paper/2511.03220/) (2025)
- [SynthSoM-Twin: A Multi-Modal Sensing-Communication Digital-Twin Dataset for Sim2Real Transfer via Synesthesia of Machines](https://mlsanity.com/paper/2511.11503/) (2025)
- [Multimodal Mixture-of-Experts for ISAC in Low-Altitude Wireless Networks](https://mlsanity.com/paper/2512.01750/) (2025)
- [GSBF: Gaussian Splatting for Environment-Aware Beamforming](https://mlsanity.com/paper/2608.05896/) (2026)
- [RadioSim Agent: Combining Large Language Models and Deterministic EM Simulators for Interactive Radio Map Analysis](https://mlsanity.com/paper/2511.05912/) (2025)
- [Small Models, Big Impact: Tool-Augmented AI Agents for Wireless Network Planning](https://mlsanity.com/paper/2601.13843/) (2026)
- [Multimodal-NF: A Wireless Dataset for Near-Field Low-Altitude Sensing and Communications](https://mlsanity.com/paper/2603.28280/) (2026)
- [LAMBDA: A Low-Altitude Multimodal Base Dataset for UAV Sensing and Communication](https://mlsanity.com/paper/2607.03826/) (2026)

## Abstract (arXiv)

Multi-modal integrated sensing and communication (ISAC) enables environmental perception and reliable connectivity for intelligent wireless networks. Data-driven multi-modal ISAC models depend heavily on annotated real-world data to learn relationships across sensing and wireless observations, thereby constraining scalable deployment. Although synthetic data generation reduces the burden, adapting existing simulation pipelines to a target deployment requires consistent scene, sensing, wireless, and learning configurations, while mismatches among these coupled components impair sim-to-real transferability. To address the challenge, we propose an agentic artificial intelligence (AI) framework for sim-to-real multi-modal ISAC, named AIMS. Given a natural-language deployment request specifying the target task, deployment conditions, and real-data budget, AIMS derives a deployment-specific sim-to-real configuration and coordinates its execution to produce a deployment-specific task model. A two-agent architecture coordinates scene construction with task learning. A scene construction agent generates geographically grounded, synchronized sensing and wireless records from shared physical states, while a scene understanding agent configures task-relevant modalities and mixture-of-experts (MoE) learning for zero-shot inference or few-shot adaptation. Structured domain knowledge guides dependency-aware planning, while validation evidence supports feedback-driven revision of affected decisions. Experiments on the real-world DeepSense 6G dataset demonstrate improved vehicle detection and beam prediction over the considered simulation and fusion baselines. A separate orchestration benchmark evaluates task interpretation, dependency reasoning, and feedback-driven replanning across diverse deployment requests, showing improved plan correctness with structured domain knowledge and validation feedback.