mlsanityData to 7 Oct 2026

Adapting LLM Agents with Universal Feedback in Communication

arXiv 2310.01444 ↗First posted 1 Oct 2023cs.CL cs.AIcited by 19

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Recent advances in large language models (LLMs) have demonstrated potential for LLM agents. To facilitate the training for these agents with both linguistic feedback and non-linguistic reward signals, we introduce Learning through Communication (LTC). We design a universal buffer to store all the feedback, and an iterative pipeline to enable an LLM agent to explore and update its policy in an given environment. To optimize agent interactions for task-specific learning with our universal buffer and pipeline, we introduce diverse communication patterns tailored for both single-agent and multi-agent environments. We evaluate the efficacy of our LTC approach on four diverse datasets: ALFWorld (single-agent), HotpotQA (multi-agent collaboration), Chameleon (multi-agent competition), and GSM8k (multi-agent teacher-student). On these data sets, LTC outperforms the supervised instruction fine-tuning baselines by 3.6% to 12%. These results highlight the versatility and efficiency of LTC in facilitating online adaptation for LLM agents.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 47 of its 64 references that are in the map. First posted on arXiv 1 Oct 2023.

Builds on

47 of 64 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Cognitive Architectures for Language Agents5 Sep
  2. 2023AgentBench: Evaluating LLMs as Agents7 Aug
  3. 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents5 Jun
  4. 2023SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks27 May

Other references · 43

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2017Proximal Policy Optimization Algorithmscited by 14,441
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  8. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  9. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  10. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  11. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  12. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
31 more
  1. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  5. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  6. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  7. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  8. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  9. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  10. 2020Learning to summarize from human feedbackcited by 2,562
  11. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  12. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  13. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  14. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  15. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  16. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  17. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  18. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  19. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  20. 2023Instruction Tuning with GPT-4cited by 676
  21. 2022PAL: Program-aided Language Modelscited by 635
  22. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 615
  23. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  24. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 395
  25. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 352
  26. 2022ScienceWorld: Is your Agent Smarter than a 5th Grader?cited by 298
  27. 2023Playing repeated games with Large Language Modelscited by 213
  28. 2023ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Modelscited by 174
  29. 2022Rationale-Augmented Ensembles in Language Modelscited by 111
  30. 2023Efficient RLHF: Reducing the Memory Usage of PPOcited by 40
  31. 2021Language Models are Few-Shot Butlerscited by 27

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 8; 2026: 5.

19 papers.

Sort:

By area:

  1. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  2. 2024When Search Engine Services meet Large Language Models: Visions and Challengescited by 59
  3. 2025Large Language Models for Planning: A Comprehensive and Systematic Surveycited by 38
  4. 2025Thought Communication in Multiagent Collaborationcited by 28
  5. 2024Cofca: A Step-Wise Counterfactual Multi-hop QA benchmarkcited by 26
  6. 2025MA-RAG: Multi-Agent Retrieval-Augmented Generation via Collaborative Chain-of-Thought Reasoningcited by 24
  7. 2024Iteration of Thought: Leveraging Inner Dialogue for Autonomous Large Language Model Reasoningcited by 16
  8. 2024Toward a Team of AI-made Scientists for Scientific Discovery from Gene Expression Datacited by 10
  9. 2025Generalizability of Large Language Model-Based Agents: A Comprehensive Surveycited by 10
  10. 2026Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directionscited by 10
  11. 2024Professional Agents -- Evolving Large Language Models into Autonomous Experts with Human-Level Competenciescited by 9
  12. 2025GenoMAS: A Multi-Agent Framework for Scientific Discovery via Code-Driven Gene Expression Analysiscited by 6
7 more
  1. 2026PABU: Progress-Aware Belief Update for Efficient LLM Agentscited by 6
  2. 2026Belief in Authority: Impact of Authority in Multi-Agent Evaluation Frameworkcited by 4
  3. 2026Supervised Fine-Tuning versus Reinforcement Learning: A Study of Post-Training Methods for Large Language Modelscited by 3
  4. 2025HERA: Hybrid Edge-cloud Resource Allocation for Cost-Efficient AI Agentscited by 1
  5. 2026Learning from Environmental Feedback: Credit Assignment across Multiple Timescales for Agentic Reinforcement Learningcited by 1
  6. 2024From Laws to Motivation: Guiding Exploration through Law-Based Reasoning and Rewardscited by 0
  7. 2025A Dynamic and High-Precision Method for Scenario-Based HRA Synthetic Data Collection in Multi-Agent Collaborative Environments Driven by LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026LLM-ALSO: LLM-Driven Adaptive Learning-Signal Optimization for Multi-Agent Reinforcement Learning
  2. 2025LLM-Agent-Controller: A Universal Multi-Agent Large Language Model System as a Control Engineer
  3. 2024Towards Efficient LLM Grounding for Embodied Multi-Agent Collaboration
  4. 2025Language-Driven Coordination and Learning in Multi-Agent Simulation Environments
  5. 2024Offline Training of Language Model Agents with Functions as Learnable Weights
  6. 2024CMAT: A Multi-Agent Collaboration Tuning Framework for Enhancing Small Language Models
  7. 2024Training Agents with Weakly Supervised Feedback from Large Language Models
  8. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMs