mlsanityData to 7 Oct 2026

Designing Fiduciary Artificial Intelligence

arXiv 2308.02435 ↗First posted 27 Jul 2023cs.CY cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

A fiduciary is a trusted agent that has the legal duty to act with loyalty and care towards a principal that employs them. When fiduciary organizations interact with users through a digital interface, or otherwise automate their operations with artificial intelligence, they will need to design these AI systems to be compliant with their duties. This article synthesizes recent work in computer science and law to develop a procedure for designing and auditing Fiduciary AI. The designer of a Fiduciary AI should understand the context of the system, identify its principals, and assess the best interests of those principals. Then the designer must be loyal with respect to those interests, and careful in an contextually appropriate way. We connect the steps in this procedure to dimensions of Trustworthy AI, such as privacy and alignment. Fiduciary AI is a promising means to address the incompleteness of data subject's consent when interacting with complex technical systems.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 40 of its 154 references that are in the map. First posted on arXiv 27 Jul 2023.

Builds on

40 of 154 references.

References · 40

  1. 2017Deep reinforcement learning from human preferencescited by 3,846
  2. 2016Concrete Problems in AI Safetycited by 1,582
  3. 2018Datasheets for Datasetscited by 1,371
  4. 2018Model Cards for Model Reportingcited by 1,068
  5. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  6. 2018Reward learning from human preferences and demonstrations in Ataricited by 314
  7. 2016Cooperative Inverse Reinforcement Learningcited by 298
  8. 2017Inverse Reward Designcited by 246
  9. 2022Power to the People? Opportunities and Challenges for Participatory AIcited by 137
  10. 2020Reward-rational (implicit) choice: A unifying formalism for reward learningcited by 119
  11. 2020Participation is not a Design Fix for Machine Learningcited by 110
  12. 2017AI Safety Gridworldscited by 108
28 more
  1. 2019Reward Tampering Problems and Solutions in Reinforcement Learning: A Causal Influence Diagram Perspectivecited by 107
  2. 2018Learning to Understand Goal Specifications by Modelling Rewardcited by 80
  3. 2021Consequences of Misaligned AIcited by 79
  4. 2019From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Followingcited by 67
  5. 2020Quantifying Differences in Reward Functionscited by 62
  6. 2020Trustworthy AIcited by 62
  7. 2018Incomplete Contracting and AI Alignmentcited by 60
  8. 2020Distributionally Robust Bayesian Optimizationcited by 60
  9. 2017A Voting-Based System for Ethical Decision Makingcited by 55
  10. 2016On the Safety of Machine Learning: Cyber-Physical Systems, Decision Sciences, and Data Productscited by 52
  11. 2021Agent Incentives: A Causal Perspectivecited by 51
  12. 2019Roles for Computing in Social Changecited by 49
  13. 2017Occam's razor is insufficient to infer the preferences of irrational agentscited by 44
  14. 2020Hidden Incentives for Auto-Induced Distributional Shiftcited by 42
  15. 2018Penalizing side effects using stepwise relative reachabilitycited by 38
  16. 2018Learning a Prior over Intent via Meta-Inverse Reinforcement Learningcited by 36
  17. 2020Participatory Problem Formulation for Fairer Machine Learning Through Community Based System Dynamicscited by 31
  18. 2020Dark Patterns after the GDPR: Scraping Consent Pop-ups and Demonstrating their Influencecited by 27
  19. 2016Engineering Safety in Machine Learningcited by 24
  20. 2018Impossibility and Uncertainty Theorems in AI Value Alignment (or why your AGI should not have a utility function)cited by 22
  21. 2022Choices, Risks, and Reward Reports: Charting Public Policy for Reinforcement Learning Systemscited by 12
  22. 2017Low Impact Artificial Intelligencescited by 11
  23. 2020Multi-Principal Assistance Gamescited by 10
  24. 2021I am Definitely Manipulated, Even When I am Aware of it. It s Ridiculous! -- Dark Patterns from the End-User Perspectivecited by 8
  25. 2020AI loyalty: A New Paradigm for Aligning Stakeholder Interestscited by 7
  26. 2021Consent Management Platforms under the GDPR: processors and/or controllers?cited by 5
  27. 2018Reinforcement Learning and Inverse Reinforcement Learning with System 1 and System 2cited by 4
  28. 2022Opted Out, Yet Tracked: Are Regulations Enough to Protect Your Privacy?cited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 3.

8 papers.

  1. 2025Infrastructure for AI Agentscited by 40
  2. 2024AI Alignment with Changing and Influenceable Reward Functionscited by 33
  3. 2025Build Agent Advocates, Not Platform Agentscited by 8
  4. 2026SovereignNegotiation-Bench: Evaluating User-Owned Personal Agents In Delegated Bargaining Under Privacy, Consent, Evidence, And Institutional Pressurecited by 1
  5. 2025From Turing to Tomorrow: The UK's Approach to AI Regulationcited by 0
  6. 2025Validity Is What You Needcited by 0
  7. 2026The Agentic Web Requires New Normative Infrastructurecited by 0
  8. 2026REDDIT: Forgetting-Resistant Correction of Timestamp Drift in ASR via Replay-Based Distribution Editingcited by 0