mlsanityData to 7 Oct 2026

MSI-Agent: Incorporating Multi-Scale Insight into Embodied Agents for Superior Planning and Decision-Making

arXiv 2409.16686 ↗First posted 25 Sep 2024cs.AI cs.CLcited by 9

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Long-term memory is significant for agents, in which insights play a crucial role. However, the emergence of irrelevant insight and the lack of general insight can greatly undermine the effectiveness of insight. To solve this problem, in this paper, we introduce Multi-Scale Insight Agent (MSI-Agent), an embodied agent designed to improve LLMs' planning and decision-making ability by summarizing and utilizing insight effectively across different scales. MSI achieves this through the experience selector, insight generator, and insight selector. Leveraging a three-part pipeline, MSI can generate task-specific and high-level insight, store it in a database, and then use relevant insight from it to aid in decision-making. Our experiments show that MSI outperforms another insight strategy when planning by GPT3.5. Moreover, We delve into the strategies for selecting seed experience and insight, aiming to provide LLM with more useful and relevant insight for better decision-making. Our observations also indicate that MSI exhibits better robustness when facing domain-shifting scenarios.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 28 of its 40 references that are in the map. First posted on arXiv 25 Sep 2024.

Builds on

28 of 40 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024PreAct: Prediction Enhances Agent's Planning Ability18 Feb
  2. 2024Investigate-Consolidate-Exploit: A General Strategy for Inter-Task Agent Self-Evolution25 Jan
  3. 2023The Rise and Potential of Large Language Model Based Agents: A Survey14 Sep
  4. 2023ExpeL: LLM Agents Are Experiential Learners20 Aug
  5. 2023AdaPlanner: Adaptive Planning from Feedback with Language Models26 May

Other references · 23

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  6. 2024Qwen2 Technical Reportcited by 2,076
  7. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  8. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  9. 2023Multimodal Foundation Models: From Specialists to General-Purpose Assistantscited by 247
  10. 2021TEACh: Task-driven Embodied Agents that Chatcited by 194
  11. 2023Synapse: Trajectory-as-Exemplar Prompting with Memory for Computer Controlcited by 180
  12. 2023Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentationcited by 146
11 more
  1. 2021A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Executioncited by 91
  2. 2023CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalizationcited by 88
  3. 2021Embodied BERT: A Transformer Model for Embodied, Language-guided Visual Task Completioncited by 57
  4. 2022Prompter: Utilizing Large Language Model Prompting for a Data Efficient Embodied Instruction Followingcited by 33
  5. 2022JARVIS: A Neuro-Symbolic Commonsense Reasoning Framework for Conversational Embodied Agentscited by 29
  6. 2022Ask4Help: Learning to Leverage an Expert for Embodied Taskscited by 18
  7. 2023Bridging the KB-Text Gap: Leveraging Structured Knowledge-aware Pre-training for KBQAcited by 17
  8. 2022DANLI: Deliberative Agent for Following Natural Language Instructionscited by 15
  9. 2023StrategyLLM: Large Language Models as Strategy Generators, Executors, Optimizers, and Evaluators for Problem Solvingcited by 13
  10. 2022Don't Copy the Teacher: Data and Model Challenges in Embodied Dialoguecited by 10
  11. 2023Interaction is all You Need? A Study of Robots Ability to Understand and Executecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 5; 2026: 2.

9 papers.

  1. 2025Evaluating LLM-based Agents for Multi-Turn Conversations: A Surveycited by 56
  2. 2024LLM-based Multi-Agent Systems: Techniques and Business Perspectivescited by 22
  3. 2025InnovatorBench: Evaluating Agents' Ability to Conduct Innovative LLM Researchcited by 18
  4. 2025RoboMemory: A Brain-inspired Multi-memory Agentic Framework for Interactive Environmental Learning in Physical Embodied Systemscited by 14
  5. 2025Agentic Knowledgeable Self-awarenesscited by 8
  6. 2025ELHPlan: Efficient Long-Horizon Task Planning for Multi-Agent Collaborationcited by 3
  7. 2026Agentic Self-Evolutionary Replanning for Embodied Navigationcited by 2
  8. 2024Scaffolded Language Models with Language Supervision for Mixed-Autonomy: A Surveycited by 1
  9. 2026Aligning Progress and Feasibility: A Neuro-Symbolic Dual Memory Framework for Long-Horizon LLM Agentscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025LLM Agents Beyond Utility: An Open-Ended Perspective
  2. 2025From Experience to Strategy: Empowering LLM Agents with Trainable Graph Memory
  3. 2026From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanisms
  4. 2025EvoMem: Improving Multi-Agent Planning with Dual-Evolving Memory
  5. 2025Coarse-to-Fine Grounded Memory for LLM Agent Planning
  6. 2024Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making
  7. 2024Unlocking the Wisdom of Large Language Models: An Introduction to The Path to Artificial General Intelligence
  8. 2026Architectural Design, Not Only Model Intelligence, Governs Multi-Agent LLM Performance