mlsanityData to 14 Oct 2026

Large Language Models as Fiduciaries: A Case Study Toward Robustly Communicating With Artificial Intelligence Through Legal Standards

arXiv 2301.10095 ↗First posted 24 Jan 2023cs.CL cs.AI cs.CYcited by 8

Not on the map. Why?

Abstract (arXiv)

Artificial Intelligence (AI) is taking on increasingly autonomous roles, e.g., browsing the web as a research assistant and managing money. But specifying goals and restrictions for AI behavior is difficult. Similar to how parties to a legal contract cannot foresee every potential "if-then" contingency of their future relationship, we cannot specify desired AI behavior for all circumstances. Legal standards facilitate robust communication of inherently vague and underspecified goals. Instructions (in the case of language models, "prompts") that employ legal standards will allow AI agents to develop shared understandings of the spirit of a directive that generalize expectations regarding acceptable actions to take in unspecified states of the world. Standards have built-in context that is lacking from other goal specification languages, such as plain language and programming languages. Through an empirical study on thousands of evaluation labels we constructed from U.S. court opinions, we demonstrate that large language models (LLMs) are beginning to exhibit an "understanding" of one of the most relevant legal standards for AI agents: fiduciary obligations. Performance comparisons across models suggest that, as LLMs continue to exhibit improved core capabilities, their legal standards understanding will also continue to improve. OpenAI's latest LLM has 78% accuracy on our data, their previous release has 73% accuracy, and a model from their 2020 GPT-3 paper has 27% accuracy (worse than random). Our research is an initial step toward a framework for evaluating AI understanding of legal standards more broadly, and for conducting reinforcement learning with legal feedback (RLLF).

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 28 of its 42 references that are in the map. First posted on arXiv 24 Jan 2023.

Builds on

28 of 42 references.

References · 28

  1. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  2. 2017Deep reinforcement learning from human preferencescited by 3,912
  3. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  4. 2020Learning to summarize from human feedbackcited by 2,588
  5. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  6. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 688
  7. 2021LexGLUE: A Benchmark Dataset for Legal Language Understanding in Englishcited by 310
  8. 2022Autoformalization with Large Language Modelscited by 263
  9. 2022The Alignment Problem from a Deep Learning Perspectivecited by 258
  10. 2017Inverse Reward Designcited by 249
  11. 2021When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Datasetcited by 201
  12. 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
16 more
  1. 2020Reward-rational (implicit) choice: A unifying formalism for reward learningcited by 120
  2. 2019Optimal Policies Tend to Seek Powercited by 119
  3. 2019Using Natural Language for Reward Shaping in Reinforcement Learningcited by 114
  4. 2022Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Datasetcited by 113
  5. 2022Is Power-Seeking AI an Existential Risk?cited by 105
  6. 2021Representation Matters: Offline Pretraining for Sequential Decision Makingcited by 91
  7. 2017Learning with Latent Languagecited by 75
  8. 2022Inferring Rewards from Language in Contextcited by 50
  9. 2019On the Feasibility of Learning, Rather than Assuming, Human Biases for Reward Inferencecited by 49
  10. 2021Grounding Language to Entities and Dynamics for Generalization in Reinforcement Learningcited by 45
  11. 2019Learning Human Objectives by Evaluating Hypothetical Behaviorcited by 41
  12. 2020Learning Rewards from Linguistic Feedbackcited by 41
  13. 2021Tell me why! Explanations support learning relational and causal structurecited by 36
  14. 2018Legible Normativity for AI Alignment: The Value of Silly Rulescited by 15
  15. 2021What About the Precedent: An Information-Theoretic Analysis of Common Lawcited by 11
  16. 2022Linguistic communication as (inverse) reward designcited by 6

Built on by

Cited by year

Citing papers per year: 2023: 5; 2024: 3.

8 papers.

  1. 2023A Short Survey of Viewing Large Language Models in Legal Aspectcited by 52
  2. 2023Large Language Models as Tax Attorneys: A Case Study in Legal Capabilities Emergencecited by 42
  3. 2024Robots in the Middle: Evaluating LLMs in Dispute Resolutioncited by 14
  4. 2023ECGBERT: Understanding Hidden Language of ECGs with Self-Supervised Representation Learningcited by 12
  5. 2023Legal Summarisation through LLMs: The PRODIGIT Projectcited by 10
  6. 2024Exploring the Nexus of Large Language Models and Legal Systems: A Short Surveycited by 6
  7. 2023Justifiable Artificial Intelligence: Engineering Large Language Models for Legal Applicationscited by 2
  8. 2024Benchmarking LLMs on the Semantic Overlap Summarization Taskcited by 2