mlsanityData to 7 Oct 2026

PolicyEvol-Agent: Evolving Policy via Environment Perception and Self-Awareness with Theory of Mind

arXiv 2504.15313 ↗First posted 20 Apr 2025cs.AI cs.LGcited by 1

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Multi-agents has exhibited significant intelligence in real-word simulations with Large language models (LLMs) due to the capabilities of social cognition and knowledge retrieval. However, existing research on agents equipped with effective cognition chains including reasoning, planning, decision-making and reflecting remains limited, especially in the dynamically interactive scenarios. In addition, unlike human, prompt-based responses face challenges in psychological state perception and empirical calibration during uncertain gaming process, which can inevitably lead to cognition bias. In light of above, we introduce PolicyEvol-Agent, a comprehensive LLM-empowered framework characterized by systematically acquiring intentions of others and adaptively optimizing irrational strategies for continual enhancement. Specifically, PolicyEvol-Agent first obtains reflective expertise patterns and then integrates a range of cognitive operations with Theory of Mind alongside internal and external perspectives. Simulation results, outperforming RL-based models and agent-based methods, demonstrate the superiority of PolicyEvol-Agent for final gaming victory. Moreover, the policy evolution mechanism reveals the effectiveness of dynamic guideline adjustments in both automatic and human evaluation.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 23 of its 42 references that are in the map. First posted on arXiv 20 Apr 2025.

Builds on

23 of 42 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Agent-Pro: Learning to Evolve via Policy-Level Reflection and Optimization27 Feb
  2. 2023Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT-429 Sep

Other references · 21

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  6. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  7. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  8. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  9. 2022SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualizationcited by 166
  10. 2016Deep Reinforcement Learning from Self-Play in Imperfect-Information Gamescited by 153
  11. 2024EvoAgent: Towards Automatic Multi-Agent Generation via Evolutionary Algorithmscited by 109
  12. 2024Large Language Models and Games: A Survey and Roadmapcited by 91
9 more
  1. 2021DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learningcited by 65
  2. 2023Boosting Theory-of-Mind Performance in Large Language Models via Promptingcited by 64
  3. 2023LLM-Based Agent Society Investigation: Collaboration and Confrontation in Avalon Gameplaycited by 62
  4. 2024NegotiationToM: A Benchmark for Stress-testing Machine Theory of Mind on Negotiation Surroundingcited by 49
  5. 2024PlanRAG: A Plan-then-Retrieval Augmented Generation for Generative Large Language Models as Decision Makerscited by 42
  6. 2019RLCard: A Toolkit for Reinforcement Learning in Card Gamescited by 34
  7. 2023COKE: A Cognitive Knowledge Graph for Machine Theory of Mindcited by 26
  8. 2023CALYPSO: LLMs as Dungeon Masters' Assistantscited by 23
  9. 2020Emergence of Pragmatics from Referential Game between Theory of Mind Agentscited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026SAGE: Structured Strategic Reasoning for Efficient LLM Game Playingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Hypothetical Minds: Scaffolding Theory of Mind for Multi-Agent Tasks with Large Language Models
  2. 2026MINDGAMES: A Live Arena for Evaluating Social and Strategic Reasoning in Multi-Agent LLMs
  3. 2026Experience-Sensitive Game Learning: A Behavioral Study of Humans and Language Agents
  4. 2025Dynamic Strategy Adaptation in Multi-Agent Environments with Large Language Models
  5. 2021Policy Fusion for Adaptive and Customizable Reinforcement Learning Agents
  6. 2026Assessing mentalization in humans and large language models
  7. 2026Readable Minds: Emergent Theory-of-Mind-Like Behavior in LLM Poker Agents
  8. 2024Evaluating and Enhancing LLMs Agent based on Theory of Mind in Guandan: A Multi-Player Cooperative Game under Imperfect Information