Dimensions of Disagreement: Unpacking Divergence and Misalignment in Cognitive Science and Artificial Intelligence
Not on the map. Why?
Abstract (arXiv)
The increasing prevalence of artificial agents creates a correspondingly increasing need to manage disagreements between humans and artificial agents, as well as between artificial agents themselves. Considering this larger space of possible agents exposes an opportunity for furthering our understanding of the nature of disagreement: past studies in psychology have often cast disagreement as two agents forming diverging evaluations of the same object, but disagreement can also arise from differences in how agents represent that object. AI research on human-machine alignment and recent work in computational cognitive science have focused on this latter kind of disagreement, and have developed tools that can be used to quantify the extent of representational overlap between agents. Understanding how divergence and misalignment interact to produce disagreement, and how resolution strategies depend on this interaction, is key to promoting effective collaboration between diverse types of agents.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 7 of its 33 references that are in the map. First posted on arXiv 3 Oct 2023.
Builds on
7 of 33 references.
References · 7
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
- 2022Relative representations enable zero-shot latent space communicationcited by 175
- 2021Geometric Deep Learning on Molecular Representationscited by 113
- 2020Understanding Human Intelligence through Human Limitationscited by 33
- 2023Alignment with human representations supports robust few-shot learningcited by 25
- 2022Words are all you need? Language as an approximation for human similarity judgmentscited by 18
Built on by
Citing papers per year: 2023: 2; 2024: 1.
3 papers.
- 2023Getting aligned on representational alignmentcited by 142
- 2023Learning Human-like Representations to Enable Learning Human Valuescited by 7
- 2024Representational Alignment Supports Effective Machine Teachingcited by 3
- 2024Representational Alignment Supports Effective Machine Teaching6 Jun 2024
- 2023Learning Human-like Representations to Enable Learning Human Values21 Dec 2023
- 2023Getting aligned on representational alignment18 Oct 2023