mlsanityData to 7 Oct 2026

Large Language Model-based Human-Agent Collaboration for Complex Task Solving

arXiv 2402.12914 ↗First posted 20 Feb 2024cs.CL cs.HCcited by 38

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

In recent developments within the research community, the integration of Large Language Models (LLMs) in creating fully autonomous agents has garnered significant interest. Despite this, LLM-based agents frequently demonstrate notable shortcomings in adjusting to dynamic environments and fully grasping human needs. In this work, we introduce the problem of LLM-based human-agent collaboration for complex task-solving, exploring their synergistic potential. In addition, we propose a Reinforcement Learning-based Human-Agent Collaboration method, ReHAC. This approach includes a policy model designed to determine the most opportune stages for human intervention within the task-solving process. We construct a human-agent collaboration dataset to train this policy model in an offline reinforcement learning environment. Our validation tests confirm the model's effectiveness. The results demonstrate that the synergistic efforts of humans and LLM-based agents significantly improve performance in complex tasks, primarily through well-planned, limited human intervention. Datasets and code are available at: https://github.com/XueyangFeng/ReHAC.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 37 of its 56 references that are in the map. First posted on arXiv 20 Feb 2024.

Builds on

37 of 56 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023TPTU-v2: Boosting Task Planning and Tool Usage of Large Language Model-based Agents in Real-world Systems19 Nov
  2. 2023Plug-and-Play Policy Planner for Large Language Model Powered Dialogue Agents1 Nov
  3. 2023Lemur: Harmonizing Natural Language and Code for Language Agents10 Oct
  4. 2023A Survey on Large Language Model based Autonomous Agents22 Aug
  5. 2023ExpeL: LLM Agents Are Experiential Learners20 Aug
  6. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework1 Aug

Other references · 31

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  8. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  9. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  10. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  11. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  12. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
19 more
  1. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  2. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  3. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  4. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  5. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 553
  6. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 496
  7. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
  8. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 424
  9. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 293
  10. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 265
  11. 2024ReFT: Reasoning with Reinforced Fine-Tuningcited by 236
  12. 2023InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedbackcited by 229
  13. 2021A Survey of Human-in-the-loop for Machine Learningcited by 214
  14. 2023Retroformer: Retrospective Large Language Agents with Policy Gradient Optimizationcited by 110
  15. 2023Drive as You Speak: Enabling Human-Like Interaction with Large Language Models in Autonomous Vehiclescited by 100
  16. 2021Putting Humans in the Natural Language Processing Loop: A Surveycited by 40
  17. 2023Improving Grounded Language Understanding in a Collaborative Environment by Interacting with Agents Through Help Feedbackcited by 31
  18. 2023Human-in-the-Loop through Chain-of-Thoughtcited by 20
  19. 2022IGLU 2022: Interactive Grounded Language Understanding in a Collaborative Environment at NeurIPS 2022cited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 17; 2026: 12.

38 papers.

Sort:

By area:

  1. 2024The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Surveycited by 164
  2. 2024Large Language Model-Brained GUI Agents: A Surveycited by 160
  3. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
  4. 2026Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligencecited by 35
  5. 2025From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Userscited by 32
  6. 2024Conformity in Large Language Modelscited by 30
  7. 2025Ambig-SWE: Interactive Agents to Overcome Underspecificity in Software Engineeringcited by 28
  8. 2025LLM-Based Human-Agent Collaboration and Interaction Systems: A Surveycited by 28
  9. 2025Multi-Agent Autonomous Driving Systems with Large Language Models: A Survey of Recent Advancescited by 21
  10. 2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skillscited by 19
  11. 2026Auditing Multi-Agent LLM Reasoning Trees Outperforms Majority Vote and LLM-as-Judgecited by 16
  12. 2024Enhancing LLM Reasoning with Multi-Path Collaborative Reactive and Reflection agentscited by 15
26 more
  1. 2026AgentHallu: Benchmarking Automated Hallucination Attribution of LLM-based Agentscited by 15
  2. 2026When Users Change Their Mind: Evaluating Interruptible Agents in Long-Horizon Web Navigationcited by 12
  3. 2024Swarm Intelligence in Geo-Localization: A Multi-Agent Large Vision-Language Model Collaborative Frameworkcited by 11
  4. 2025Invisible Tokens, Visible Bills: The Urgent Need to Audit Hidden Operations in Opaque LLM Servicescited by 10
  5. 2026The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why -- A Survey from MARL to Emergent Language and LLMscited by 10
  6. 2024Preemptive Detection and Correction of Misaligned Actions in LLM Agentscited by 9
  7. 2025A Call for Collaborative Intelligence: Why Human-Agent Systems Should Precede AI Autonomycited by 9
  8. 2026Adaptive Collaboration with Humans: Metacognitive Policy Optimization for Multi-Agent LLMs with Continual Learningcited by 9
  9. 2024Nadine: An LLM-driven Intelligent Social Robot with Affective Capabilities and Human-like Memorycited by 6
  10. 2024Zero-shot Model-based Reinforcement Learning using Large Language Modelscited by 6
  11. 2025Chandomitra: Towards Generating Structured Sanskrit Poetry from Natural Language Inputscited by 5
  12. 2025Embracing Trustworthy Brain-Agent Collaboration as Paradigm Extension for Intelligent Assistive Technologiescited by 5
  13. 2025How to Enable Effective Cooperation Between Humans and NLP Models: A Survey of Principles, Formalizations, and Beyondcited by 4
  14. 2026Agent Skills Matter: Inferring Proprietary Skills from Execution Trajectoriescited by 4
  15. 2024Physics-Informed Autonomous LLM Agents for Explainable Power Electronics Modulation Designcited by 3
  16. 2025Adaptive Command: Real-Time Policy Adjustment via Language Models in StarCraft IIcited by 3
  17. 2025Understanding Post-Training Structural Changes in Large Language Modelscited by 2
  18. 2025ProMediate: A Socio-cognitive framework for evaluating proactive agents in multi-party negotiationcited by 2
  19. 2025Towards Unification of Hallucination Detection and Fact Verification for Large Language Modelscited by 2
  20. 2026Harnesses for Inference-Time Alignment over Execution Trajectoriescited by 2
  21. 2025Mixed-Initiative Dialog for Human-Robot Collaborative Manipulationcited by 1
  22. 2026Imperfectly Cooperative Human-AI Interactions: Comparing the Impacts of Human and AI Attributes in Simulated and User Studiescited by 1
  23. 2025What's the next frontier for Data-centric AI? Data Savvy Agentscited by 0
  24. 2026More than a Judge: An Empirical Study of Agent-Human Interaction in Crowdsourced Testing Assessmentcited by 0
  25. 2026Toward Human-Centered Multi-Agent Systems: Integrating Cognition, Culture, Values, and Cooperation in AI Agentscited by 0
  26. 2026Turning Intent into Specifications: A Benchmark and an Interactive User-Assistant Agentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Synergistic Simulations: Multi-Agent Problem Solving with Large Language Models
  2. 2025Enhancing Multi-Agent Systems via Reinforcement Learning with LLM-based Planner and Graph-based Policy
  3. 2025Multi-Agent Language Models: Advancing Cooperation, Coordination, and Adaptation
  4. 2023Theory of Mind for Multi-Agent Collaboration via Large Language Models
  5. 2024Two Heads Are Better Than One: Collaborative LLM Embodied Agents for Human-Robot Interaction
  6. 2026Rethinking Agentic Reinforcement Learning In Large Language Models
  7. 2025Reinforcement Learning-Augmented LLM Agents for Collaborative Decision Making and Performance Optimization
  8. 2025Can LLM Agents Solve Collaborative Tasks? A Study on Urgency-Aware Planning and Coordination