mlsanityData to 7 Oct 2026

IoT-Brain: Grounding LLMs for Semantic-Spatial Sensor Scheduling

arXiv 2604.08033 ↗First posted 9 Apr 2026cs.AI cs.MA cs.NIcited by 2

Not on the map. Why?

Abstract (arXiv)

Intelligent systems powered by large-scale sensor networks are shifting from predefined monitoring to intent-driven operation, revealing a critical Semantic-to-Physical Mapping Gap. While large language models (LLMs) excel at semantic understanding, existing perception-centric pipelines operate retrospectively, overlooking the fundamental decision of what to sense and when. We formalize this proactive decision as Semantic-Spatial Sensor Scheduling (S3) and demonstrate that direct LLM planning is unreliable due to inherent gaps in representation, reasoning, and optimization. To bridge these gaps, we introduce the Spatial Trajectory Graph (STG), a neuro-symbolic paradigm governed by a verify-before-commit discipline that transforms open-ended planning into a verifiable graph optimization problem. Based on STG, we implement IoT-Brain, a concrete system embodiment, and construct TopoSense-Bench, a campus-scale benchmark with 5,250 natural-language queries across 2,510 cameras. Evaluations show that IoT-Brain boosts task success rate by 37.6% over the strongest search-intensive methods while running nearly 2 times faster and using 6.6 times fewer prompt tokens. In real-world deployment, it approaches the reliability upper bound while reducing 4.1 times network bandwidth, providing a foundational framework for LLMs to interact with the physical world with unprecedented reliability and efficiency.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 32 of its 77 references that are in the map. First posted on arXiv 9 Apr 2026.

Builds on

32 of 77 references.

References · 32

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  5. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  6. 2015VQA: Visual Question Answeringcited by 2,489
  7. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  8. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  9. 2024A Survey on Large Language Models for Code Generationcited by 646
  10. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 559
  11. 2019Omni-Scale Feature Learning for Person Re-Identificationcited by 292
  12. 2024Tool Learning with Large Language Models: A Surveycited by 221
20 more
  1. 2023OpenAGI: When LLM Meets Domain Expertscited by 196
  2. 2019CityFlow: A City-Scale Benchmark for Multi-Target Multi-Camera Vehicle Tracking and Re-Identificationcited by 119
  3. 2022MeMOT: Multi-Object Tracking with Memorycited by 113
  4. 2021Structured Scene Memory for Vision-Language Navigationcited by 95
  5. 2024Plan-on-Graph: Self-Correcting Adaptive Planning of Large Language Model on Knowledge Graphscited by 77
  6. 2019Learning Generalisable Omni-Scale Representations for Person Re-Identificationcited by 73
  7. 2023Penetrative AI: Making LLMs Comprehend the Physical Worldcited by 64
  8. 2023ControlLLM: Augment Language Models with Tools by Searching on Graphscited by 59
  9. 2023Fairness-guided Few-shot Prompting for Large Language Modelscited by 53
  10. 2024Harnessing the Power of MLLMs for Transferable Text-to-Image Person ReIDcited by 39
  11. 2022Learning to Answer Visual Questions from Web Videoscited by 30
  12. 2025UrbanLLaVA: A Multi-modal Large Language Model for Urban Intelligence with Spatial Reasoning and Understandingcited by 15
  13. 2024CATP-LLM: Empowering Large Language Models for Cost-Aware Tool Planningcited by 14
  14. 2024Smurfs: Multi-Agent System using Context-Efficient DFSDT for Tool Planningcited by 12
  15. 2025SensorChat: Answering Qualitative and Quantitative Questions during Long-Term Multimodal Sensor Interactionscited by 11
  16. 2025GPIoT: Tailoring Small Language Models for IoT Program Synthesis and Developmentcited by 10
  17. 2024LaMOT: Language-Guided Multi-Object Trackingcited by 9
  18. 2025Anticipate & Act : Integrating LLMs and Classical Planning for Efficient Task Execution in Household Environmentscited by 5
  19. 2021Graph-Based Topological Exploration Planning in Large-Scale 3D Environmentscited by 4
  20. 2020Adaptation to Team Composition Changes for Heterogeneous Multi-Robot Sensor Coveragecited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.