mlsanityData to 7 Oct 2026

Agent Alignment in Evolving Social Norms

arXiv 2401.04620 ↗First posted 9 Jan 2024cs.CL cs.AIcited by 14

On the map: LLM agents and tool use → Social simulation and multi-agent debate

also in 1 more
Abstract (arXiv)

Agents based on Large Language Models (LLMs) are increasingly permeating various domains of human production and life, highlighting the importance of aligning them with human values. The current alignment of AI systems primarily focuses on passively aligning LLMs through human intervention. However, agents possess characteristics like receiving environmental feedback and self-evolution, rendering the LLM alignment methods inadequate. In response, we propose an evolutionary framework for agent evolution and alignment, named EvolutionaryAgent, which transforms agent alignment into a process of evolution and selection under the principle of survival of the fittest. In an environment where social norms continuously evolve, agents better adapted to the current social norms will have a higher probability of survival and proliferation, while those inadequately aligned dwindle over time. Experimental results assessing the agents from multiple perspectives in aligning with social norms demonstrate that EvolutionaryAgent can align progressively better with the evolving social norms while maintaining its proficiency in general tasks. Effectiveness tests conducted on various open and closed-source LLMs as the foundation for agents also prove the applicability of our approach.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 43 of its 68 references that are in the map. First posted on arXiv 9 Jan 2024.

Builds on

43 of 68 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Generative Agents: Interactive Simulacra of Human Behavior7 Apr

Other references · 42

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  9. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  10. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  11. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  12. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
30 more
  1. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  2. 2020Learning to summarize from human feedbackcited by 2,562
  3. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  4. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  5. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  6. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  7. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  8. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  9. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  10. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  11. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 615
  12. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  13. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
  14. 2020Artificial Intelligence, Values and Alignmentcited by 443
  15. 2023Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervisioncited by 318
  16. 2022Generating Sequences by Learning to Self-Correctcited by 271
  17. 2020Social Chemistry 101: Learning to Reason about Social and Moral Normscited by 253
  18. 2023Large Language Model Alignment: A Surveycited by 241
  19. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 236
  20. 2022Measuring Progress on Scalable Oversight for Large Language Modelscited by 212
  21. 2023JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Modelscited by 170
  22. 2021Alignment of Language Agentscited by 167
  23. 2023Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generationcited by 128
  24. 2023A Definition of Continual Reinforcement Learningcited by 98
  25. 2023From Instructions to Intrinsic Human Values -- A Survey of Alignment Goals for Big Modelscited by 45
  26. 2023SelfEvolve: A Code Evolution Framework via Large Language Modelscited by 43
  27. 2023Large Language Models Are Semi-Parametric Reinforcement Learning Agentscited by 42
  28. 2023Natural Selection Favors AIs over Humanscited by 24
  29. 2023ALAN: Autonomously Exploring Robotic Agents in the Real Worldcited by 23
  30. 2023MAF: Multi-Aspect Feedback for Improving Reasoning in Large Language Modelscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 5; 2026: 4.

14 papers.

Sort:

By area:

  1. 2024Emergent social conventions and collective bias in LLM populationscited by 115
  2. 2025Agentic Large Language Models, a surveycited by 105
  3. 2024A Survey on Self-Evolution of Large Language Modelscited by 86
  4. 2025From Automation to Autonomy: A Survey on Large Language Models in Scientific Discoverycited by 86
  5. 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 74
  6. 2024Emergence of Social Norms in Generative Agent Societies: Principles and Architecturecited by 41
  7. 2024Assistive Large Language Model Agents for Socially-Aware Negotiation Dialoguescited by 18
  8. 2025Virtual Agent Economiescited by 17
  9. 2025The Role of Social Learning and Collective Norm Formation in Fostering Cooperation in LLM Multi-Agent Systemscited by 8
  10. 2024Situated Instruction Followingcited by 3
  11. 2026Social Norm Reasoning in Multimodal Language Models: An Evaluationcited by 1
  12. 2026Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Designcited by 1
2 more
  1. 2026Group Selection Promotes Prosocial Prompts in Populations of LLM Agentscited by 0
  2. 2026AI Alignment through a Game-theoretic Lens: A Surveycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026AI Value Alignment for Evolving Social Norms
  2. 2026Evolving Interpretable Constitutions for Multi-Agent Coordination
  3. 2026Toward Human-Centered Multi-Agent Systems: Integrating Cognition, Culture, Values, and Cooperation in AI Agents
  4. 2024Cultural Evolution of Cooperation among LLM Agents
  5. 2024Evolution of Social Norms in LLM Agents using Natural Language
  6. 2025The Coming Crisis of Multi-Agent Misalignment: AI Alignment Must Be a Dynamic and Social Process
  7. 2025Evolution of Cooperation in LLM-Agent Societies: A Preliminary Study Using Different Punishment Strategies
  8. 2026SODE: Analyzing Social Dynamics in LLM Agents