mlsanityData to 7 Oct 2026

IoT-LLM: a framework for enhancing Large Language Model reasoning from real-world sensor data

arXiv 2410.02429 ↗First posted 3 Oct 2024cs.AI cs.CLcited by 18

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large Language Models excel in textual tasks but often struggle with physical-world reasoning tasks. Inspired by human cognition, where perception is fundamental to reasoning, we explore augmenting LLMs with enhanced perception abilities using Internet of Things (IoT) data and pertinent knowledge. In this work, we systematically study LLMs' capability to address IoT-sensory tasks by augmenting their perception and knowledge base, and then propose a unified framework, IoT-LLM, to enhance such capability. In IoT-LLM, we customize three steps: preprocessing IoT data into suitable formats, expanding LLMs knowledge via IoT-oriented retrieval-augmented generation and activating LLMs commonsense knowledge through chain-of-thought prompting. We design a benchmark comprising five real-world tasks with varying data types and reasoning complexities to evaluate the performance of IoT-LLM. Experimental results reveal that IoT-LLM significantly improves the performance of IoT-sensory task reasoning of LLMs, with models like GPT-4o-mini showing a 49.4% average improvement over previous methods.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 31 of its 70 references that are in the map. First posted on arXiv 3 Oct 2024.

Builds on

31 of 70 references.

References · 31

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  7. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  8. 2022Scalable Diffusion Models with Transformerscited by 5,931
  9. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  10. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  11. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  12. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
19 more
  1. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  2. 2019Searching for MobileNetV3cited by 1,984
  3. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  4. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  5. 2019RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Cloudscited by 584
  6. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
  7. 2023EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thoughtcited by 276
  8. 2023OneLLM: One Framework to Align All Modalities with Languagecited by 185
  9. 2023Meta-Transformer: A Unified Framework for Multimodal Learningcited by 136
  10. 2024Large Language Models for Time Series: A Surveycited by 113
  11. 2024Tokenization counts: the impact of tokenization on arithmetic in frontier LLMscited by 112
  12. 2024Large Language Models for Robotics: Opportunities, Challenges, and Perspectivescited by 88
  13. 2023MM-Fi: Multi-Modal Non-Intrusive 4D Human Dataset for Versatile Wireless Sensingcited by 77
  14. 2024Learning and Leveraging World Models in Visual Representation Learningcited by 71
  15. 2023Penetrative AI: Making LLMs Comprehend the Physical Worldcited by 64
  16. 2023Introduction to Latent Variable Energy-Based Models: A Path Towards Autonomous Machine Intelligencecited by 52
  17. 2024HARGPT: Are LLMs Zero-Shot Human Activity Recognizers?cited by 52
  18. 2023The first step is the hardest: Pitfalls of Representing and Tokenizing Temporal Data for Large Language Modelscited by 34
  19. 2024Multi-Meta-RAG: Improving RAG for Multi-Hop Queries using Database Filtering with LLM-Extracted Metadatacited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 7; 2026: 9.

18 papers.

Sort:

By area:

  1. 2024When IoT Meet LLMs: Applications and Challengescited by 11
  2. 2025REI-Bench: Can Embodied Agents Understand Vague Human Instructions in Task Planning?cited by 7
  3. 2025Enhancing LLM Reasoning for Time Series Classification by Tailored Thinking and Fused Decisioncited by 7
  4. 2025DailyLLM: Context-Aware Activity Log Generation Using Multi-Modal Sensors and LLMscited by 5
  5. 2024A Unified Framework for Context-Aware IoT Management and State-of-the-Art IoT Traffic Anomaly Detectioncited by 2
  6. 2026ThermoLLM: Thermodynamics-Aware HVAC Control with Spatial-Semantic Knowledge Graphcited by 2
  7. 2025SignalLLM: A General-Purpose LLM Agent Framework for Automated Signal Processingcited by 1
  8. 2026Proactive Rejection and Grounded Execution: A Dual-Stage Intent Analysis Paradigm for Safe and Efficient AIoT Smart Homescited by 1
  9. 2025Intelligence of Things: A Spatial Context-Aware Control System for Smart Devicescited by 0
  10. 2025MARCO: Meta-Reflection with Cross-Referencing for Code Reasoningcited by 0
  11. 2025RadarPLM: Adapting Pre-trained Language Models for Marine Radar Target Detection by Selective Fine-tuningcited by 0
  12. 2026SensingAgents: A Multi-Agent Collaborative Framework for Robust IMU Activity Recognitioncited by 0
6 more
  1. 2026PULSE: Agentic Investigation with Passive Sensing for Proactive Affective Intervention in Cancer Survivorshipcited by 0
  2. 2026Enabling Cloud-Level Accuracy in Edge AI through IoT Data Preprocessingcited by 0
  3. 2026Design and Implementation of Agentic Orchestrations and Orchestration of Agentscited by 0
  4. 2026Persona-as-Configuration: Generative Stakeholder Reporting for Agricultural Floodscited by 0
  5. 2026Benchmarking Large Language Models on Multi-Sensor Physical Hazard Assessmentcited by 0
  6. 2026SenseAgent: An LLM Agent for Adaptive Cross-Domain IMU Sensingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Talk with the Things: Integrating LLMs into IoT Networks ↗
  2. 2024LLMSense: Harnessing LLMs for High-level Reasoning Over Spatiotemporal Sensor Traces
  3. 2025LLMind 2.0: Distributed IoT Automation with Natural Language M2M Communication and Lightweight LLM Agents
  4. 2025IoT-MCP: Bridging LLMs and IoT Systems Through Model Context Protocol
  5. 2024Iteration of Thought: Leveraging Inner Dialogue for Autonomous Large Language Model Reasoning
  6. 2026SMH-Bench: Benchmarking LLM Agents for Environment-Grounded Reasoning and Action in Smart Homes
  7. 2026The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modes
  8. 2025Fast, Slow, and Tool-augmented Thinking for LLMs: A Review