mlsanityData to 14 Oct 2026

LA-RCS: LLM-Agent-Based Robot Control System

arXiv 2505.18214 ↗First posted 23 May 2025cs.RO cs.AIcited by 1

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

LA-RCS (LLM-agent-based robot control system) is a sophisticated robot control system designed to autonomously plan, work, and analyze the external environment based on user requirements by utilizing LLM-Agent. Utilizing a dual-agent framework, LA-RCS generates plans based on user requests, observes the external environment, executes the plans, and modifies the plans as needed to adapt to changes in the external conditions. Additionally, LA-RCS interprets natural language commands by the user and converts them into commands compatible with the robot interface so that the robot can execute tasks and meet user requests properly. During his process, the system autonomously evaluates observation results, provides feedback on the tasks, and executes commands based on real-time environmental monitoring, significantly reducing the need for user intervention in fulfilling requests. We categorized the scenarios that LA-RCS needs to perform into four distinct types and conducted a quantitative assessment of its performance in each scenario. The results showed an average success rate of 90 percent, demonstrating the system capability to fulfill user requests satisfactorily. For more extensive results, readers can visit our project page: https://la-rcs.github.io

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 44 of its 52 references that are in the map. First posted on arXiv 23 May 2025.

Builds on

44 of 52 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challenges21 Jan
  2. 2023AutoAgents: A Framework for Automatic Agent Generation29 Sep
  3. 2023SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models18 Sep
  4. 2023The Rise and Potential of Large Language Model Based Agents: A Survey14 Sep
  5. 2023A Survey on Large Language Model based Autonomous Agents22 Aug
  6. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework1 Aug
  7. 2023AdaPlanner: Adaptive Planning from Feedback with Language Models26 May

Other references · 37

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  7. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  8. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  9. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  10. 2022Emergent Abilities of Large Language Modelscited by 2,512
  11. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  12. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
25 more
  1. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  2. 2023A Comprehensive Overview of Large Language Modelscited by 648
  3. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 611
  4. 2024GPT-4V(ision) is a Generalist Web Agent, if Groundedcited by 563
  5. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 478
  6. 2023AppAgent: Multimodal Agents as Smartphone Userscited by 415
  7. 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Casescited by 393
  8. 2023ChatGPT for Robotics: Design Principles and Model Abilitiescited by 299
  9. 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perceptioncited by 293
  10. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 233
  11. 2023Text2Motion: From Natural Language Instructions to Feasible Planscited by 224
  12. 2023OpenAGI: When LLM Meets Domain Expertscited by 197
  13. 2024UFO: A UI-Focused Agent for Windows OS Interactioncited by 148
  14. 2023GPT-4V(ision) as a Generalist Evaluator for Vision-Language Taskscited by 119
  15. 2024AIOS: LLM Agent Operating Systemcited by 97
  16. 2024Large Language Models for Robotics: Opportunities, Challenges, and Perspectivescited by 89
  17. 2023TaskWeaver: A Code-First Agent Frameworkcited by 86
  18. 2023GPT-4V(ision) for Robotics: Multimodal Task Planning from Human Demonstrationcited by 77
  19. 2023Language Models as Zero-Shot Trajectory Generatorscited by 60
  20. 2024Harnessing Large Language Models for Text-Rich Sequential Recommendationcited by 38
  21. 2024Exploring the Capabilities and Limitations of Large Language Models in the Electric Energy Sectorcited by 30
  22. 2024ROS-LLM: A ROS framework for embodied AI with task feedback and structured reasoningcited by 27
  23. 2024Enhancing the LLM-Based Robot Manipulation Through Human-Robot Collaborationcited by 19
  24. 2023LLM-Based Human-Robot Collaboration Framework for Manipulation Taskscited by 17
  25. 2022Multimodal Dialog Systems with Dual Knowledge-enhanced Generative Pretrained Language Modelcited by 11

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Embodied4C: Measuring What Matters for Embodied Vision-Language Navigationcited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2024LLM2Swarm: Robot Swarms that Responsively Reason, Plan, and Collaborate through LLMs
  2. 2025Multi-Agent Systems for Robotic Autonomy with LLMs
  3. 2025LLM-Agent-Controller: A Universal Multi-Agent Large Language Model System as a Control Engineer
  4. 2025Pretrained LLMs as Real-Time Controllers for Robot Operated Serial Production Line
  5. 2025AuDeRe: Automated Strategy Decision and Realization in Robot Planning and Control via LLMs
  6. 2025LEO-RobotAgent: A General-purpose Robotic Agent for Language-driven Embodied Operator
  7. 2025LLM-Based Human-Agent Collaboration and Interaction Systems: A Survey
  8. 2025Exploring GPT-4 for Robotic Agent Strategy with Real-Time State Feedback and a Reactive Behaviour Framework