mlsanityData to 14 Oct 2026

Getting In Contract with Large Language Models -- An Agency Theory Perspective On Large Language Model Alignment

arXiv 2509.07642 ↗First posted 9 Sep 2025cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Adopting Large language models (LLMs) in organizations potentially revolutionizes our lives and work. However, they can generate off-topic, discriminating, or harmful content. This AI alignment problem often stems from misspecifications during the LLM adoption, unnoticed by the principal due to the LLM's black-box nature. While various research disciplines investigated AI alignment, they neither address the information asymmetries between organizational adopters and black-box LLM agents nor consider organizational AI adoption processes. Therefore, we propose LLM ATLAS (LLM Agency Theory-Led Alignment Strategy) a conceptual framework grounded in agency (contract) theory, to mitigate alignment problems during organizational LLM adoption. We conduct a conceptual literature analysis using the organizational LLM adoption phases and the agency theory as concepts. Our approach results in (1) providing an extended literature analysis process specific to AI alignment methods during organizational LLM adoption and (2) providing a first LLM alignment problem-solution space.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 44 of its 72 references that are in the map. First posted on arXiv 9 Sep 2025.

Builds on

44 of 72 references.

References · 44

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2017Proximal Policy Optimization Algorithmscited by 15,215
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  7. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  8. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  9. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  10. 2017Deep reinforcement learning from human preferencescited by 3,912
  11. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  12. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
32 more
  1. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  2. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  3. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  4. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  5. 2023PaLM 2 Technical Reportcited by 1,162
  6. 2023LIMA: Less Is More for Alignmentcited by 1,100
  7. 2018Model Cards for Model Reportingcited by 1,080
  8. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
  9. 2021Ethical and social risks of harm from Language Modelscited by 966
  10. 2023BloombergGPT: A Large Language Model for Financecited by 939
  11. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 763
  12. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
  13. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 686
  14. 2024Large Language Models: A Surveycited by 525
  15. 2020Artificial Intelligence, Values and Alignmentcited by 448
  16. 2021A Survey on Automated Fact-Checkingcited by 378
  17. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 376
  18. 2023Role-Play with Large Language Modelscited by 294
  19. 2023AI Alignment: A Comprehensive Surveycited by 263
  20. 2023Large Language Model Alignment: A Surveycited by 243
  21. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 223
  22. 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
  23. 2023ChipNeMo: Domain-Adapted LLMs for Chip Designcited by 174
  24. 2022Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AIcited by 171
  25. 2021Alignment of Language Agentscited by 168
  26. 2023Auditing large language models: a three-layered approachcited by 137
  27. 2022Machine Learning Operations (MLOps): Overview, Definition, and Architecturecited by 128
  28. 2023Generative AIcited by 120
  29. 2017Ethical Challenges in Data-Driven Dialogue Systemscited by 86
  30. 2018Incomplete Contracting and AI Alignmentcited by 62
  31. 2022MLOps -- Definitions, Tools and Challengescited by 25
  32. 2023The Costly Dilemma: Generalization, Evaluation and Cost-Optimal Deployment of Large Language Modelscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Dynamic Alignment for Collective Agency: Toward a Scalable Self-Improving Framework for Open-Ended LLM Alignment
  2. 2025Societal Alignment Frameworks Can Improve LLM Alignment
  3. 2025Evaluating Intra-firm LLM Alignment Strategies in Business Contexts
  4. 2025'Too much alignment; not enough culture': Re-balancing cultural alignment practices in LLMs
  5. 2026Toward an Organizational Science of Multi-Agent LLM Systems: Decoupling Who, How, and Which Algorithm
  6. 2025Decoding Alignment: A Critical Survey of LLM Development Initiatives through Value-setting and Data-centric Lens
  7. 2025Evaluating AI Alignment in LLMs: Output Analysis of Value Priorities Across 75 Models with Human Benchmarking
  8. 2025Decentralising LLM Alignment: A Case for Context, Pluralism, and Participation