mlsanityData to 13 Oct 2026

Interpretable Locomotion Prediction in Construction Using a Memory-Driven LLM Agent With Chain-of-Thought Reasoning

arXiv 2504.15263 ↗First posted 21 Apr 2025cs.ROcited by 0

On the map: LLM reasoning → Spatial and embodied reasoning

also in 1 more
Abstract (arXiv)

Construction tasks are inherently unpredictable, with dynamic environments and safety-critical demands posing significant risks to workers. Exoskeletons offer potential assistance but falter without accurate intent recognition across diverse locomotion modes. This paper presents a locomotion prediction agent leveraging Large Language Models (LLMs) augmented with memory systems, aimed at improving exoskeleton assistance in such settings. Using multimodal inputs - spoken commands and visual data from smart glasses - the agent integrates a Perception Module, Short-Term Memory (STM), Long-Term Memory (LTM), and Refinement Module to predict locomotion modes effectively. Evaluation reveals a baseline weighted F1-score of 0.73 without memory, rising to 0.81 with STM, and reaching 0.90 with both STM and LTM, excelling with vague and safety-critical commands. Calibration metrics, including a Brier Score drop from 0.244 to 0.090 and ECE from 0.222 to 0.044, affirm improved reliability. This framework supports safer, high-level human-exoskeleton collaboration, with promise for adaptive assistive systems in dynamic industries.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 9 of its 46 references that are in the map. First posted on arXiv 21 Apr 2025.

Builds on

9 of 46 references.

References · 9

  1. 2020Language Models are Few-Shot Learnerscited by 32,320
  2. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,765
  3. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,197
  4. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  5. 2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 292
  6. 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problemscited by 258
  7. 2023RecMind: Large Language Model Powered Agent For Recommendationcited by 129
  8. 2023AgentSims: An Open-Source Sandbox for Large Language Model Evaluationcited by 87
  9. 2023MetaAgents: Large Language Model Based Agents for Decision-Making on Teamingcited by 85

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026LocoVLM: Grounding Vision and Language for Adapting Versatile Legged Locomotion Policies
  2. 2024Robots Can Multitask Too: Integrating a Memory Architecture and LLMs for Enhanced Cross-Task Robot Action Generation
  3. 2026From Language to Action: Can LLM-Based Agents Be Used for Embodied Robot Cognition?
  4. 2024LLM3:Large Language Model-based Task and Motion Planning with Motion Failure Reasoning
  5. 2026HarnessVLN: Unifying Training-Free Embodied Navigation through an Agent Harness
  6. 2026STEP: State-Aware Task Estimation and Planning with Multi-Modal LLMs for Human-Robot Collaboration
  7. 2023GG-LLM: Geometrically Grounding Large Language Models for Zero-shot Human Activity Forecasting in Human-Aware Task Planning
  8. 2025From Experience to Strategy: Empowering LLM Agents with Trainable Graph Memory