mlsanityData to 7 Oct 2026

Concept Alignment

arXiv 2401.08672 ↗First posted 9 Jan 2024cs.LG cs.AI q-bio.NCcited by 9

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Discussion of AI alignment (alignment between humans and AI systems) has focused on value alignment, broadly referring to creating AI systems that share human values. We argue that before we can even attempt to align values, it is imperative that AI systems and humans align the concepts they use to understand the world. We integrate ideas from philosophy, cognitive science, and deep learning to explain the need for concept alignment, not just value alignment, between humans and machines. We summarize existing accounts of how humans and machines currently learn concepts, and we outline opportunities and challenges in the path towards shared concepts. Finally, we explain how we can leverage the tools already being developed in cognitive science and AI research to accelerate progress towards concept alignment.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 28 of its 67 references that are in the map. First posted on arXiv 9 Jan 2024.

Builds on

28 of 67 references.

Closest ancestors · 1

Same direction, newest first

  1. 2020Aligning AI With Shared Human Values5 Aug

Other references · 27

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  6. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  8. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  9. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  10. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  11. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  12. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
15 more
  1. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  2. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  3. 2018Sanity Checks for Saliency Mapscited by 1,149
  4. 2016Adversarial Machine Learning at Scalecited by 934
  5. 2018Robustness May Be at Odds with Accuracycited by 873
  6. 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 637
  7. 2020Artificial Intelligence, Values and Alignmentcited by 443
  8. 2016Cooperative Inverse Reinforcement Learningcited by 298
  9. 2017Inverse Reward Designcited by 246
  10. 2023The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networkscited by 153
  11. 2019On the Connection Between Adversarial Robustness and Saliency Map Interpretabilitycited by 80
  12. 2016Dialogue Learning With Human-In-The-Loopcited by 46
  13. 2015Take and Took, Gaggle and Goose, Book and Read: Evaluating the Utility of Vector Differences for Lexical Relation Learningcited by 29
  14. 2021From partners to populations: A hierarchical Bayesian account of coordination and conventioncited by 27
  15. 2021Shades of confusion: Lexical uncertainty modulates ad hoc coordination in an interactive communication taskcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 3; 2026: 3.

9 papers.

  1. 2024Editing Conceptual Knowledge for Large Language Modelscited by 21
  2. 2024Abstraction Alignment: Comparing Model-Learned and Human-Encoded Conceptual Relationshipscited by 7
  3. 2025On the Inevitability of Left-Leaning Political Bias in Aligned Language Modelscited by 3
  4. 2025The Alignment Bottleneckcited by 1
  5. 2024The Reasonable Person Standard for AIcited by 0
  6. 2025Concept Concentration for Faithful Representation Interventioncited by 0
  7. 2026Investigating Concept Alignment Using Implausible Category Memberscited by 0
  8. 2026A Taxonomy of Conceptual Alignment in Human-Robot Dialoguecited by 0
  9. 2026Alignment Games: A Framework for Conceptual Repair in Human-AI Collaborationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Position: Towards Bidirectional Human-AI Alignment
  2. 2023Concept Alignment as a Prerequisite for Value Alignment
  3. 2026Toward a Theory of Value in AI Alignment
  4. 2024ValueCompass: A Framework for Measuring Contextual Value Alignment Between Human and LLMs
  5. 2023A Multi-Level Framework for the AI Alignment Problem
  6. 2024The AI Alignment Paradox
  7. 2024Dynamic Normativity: Necessary and Sufficient Conditions for Value Alignment
  8. 2025Human-AI Interaction Alignment: Designing, Evaluating, and Evolving Value-Centered AI For Reciprocal Human-AI Futures