mlsanityData to 7 Oct 2026

In conversation with Artificial Intelligence: aligning language models with human values

arXiv 2209.00731 ↗First posted 1 Sep 2022cs.CY cs.CLcited by 61

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Large-scale language technologies are increasingly used in various forms of communication with humans across different contexts. One particular use case for these technologies is conversational agents, which output natural language text in response to prompts and queries. This mode of engagement raises a number of social and ethical questions. For example, what does it mean to align conversational agents with human norms or values? Which norms or values should they be aligned with? And how can this be accomplished? In this paper, we propose a number of steps that help answer these questions. We start by developing a philosophical analysis of the building blocks of linguistic communication between conversational agents and human interlocutors. We then use this analysis to identify and formulate ideal norms of conversation that can govern successful linguistic communication between humans and conversational agents. Furthermore, we explore how these norms can be used to align conversational agents with human values across a range of different discursive domains. We conclude by discussing the practical implications of our proposal for the design of conversational agents that are aligned with these norms and values.

Key facts. Cited by 61 papers in the map (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 24 of its 135 references that are in the map. First posted on arXiv 1 Sep 2022.

Builds on

24 of 135 references.

References · 24

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  6. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  7. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  8. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  9. 2021Ethical and social risks of harm from Language Modelscited by 957
  10. 2020Artificial Intelligence, Values and Alignmentcited by 443
  11. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 376
  12. 2016A Network-based End-to-End Trainable Task-oriented Dialogue Systemcited by 252
12 more
  1. 2018Machine Theory of Mindcited by 222
  2. 2021Challenges in Detoxifying Language Modelscited by 169
  3. 2021Alignment of Language Agentscited by 167
  4. 2021Understanding the Capabilities, Limitations, and Societal Impact of Large Language Modelscited by 159
  5. 2021Truthful AI: Developing and governing AI that does not liecited by 109
  6. 2017Ethical Challenges in Data-Driven Dialogue Systemscited by 86
  7. 2020CPM: A Large-scale Generative Chinese Pre-trained Language Modelcited by 60
  8. 2021The Challenge of Value Alignment: from Fairer Algorithms to AI Safetycited by 34
  9. 2021Envisioning Communities: A Participatory Approach Towards AI for Social Goodcited by 29
  10. 2021Rethinking Search: Making Domain Experts out of Dilettantescited by 18
  11. 2022Algorithmic Fairness and Structural Injustice: Insights from Feminist Political Philosophycited by 15
  12. 2021Toward a Theory of Justice for Artificial Intelligencecited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 13; 2024: 18; 2025: 20; 2026: 8.

61 papers.

Sort:

By area:

  1. 2022Editing Models with Task Arithmeticcited by 1,076
  2. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 424
  3. 2023Knowledge Editing for Large Language Models: A Surveycited by 189
  4. 2023Auditing large language models: a three-layered approachcited by 135
  5. 2024Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Modelscited by 122
  6. 2023Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensionscited by 111
  7. 2023Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedbackcited by 86
  8. 2024Strengthening Multimodal Large Language Model with Bootstrapped Preference Optimizationcited by 75
  9. 2024Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapescited by 63
  10. 2024Beyond Preferences in AI Alignmentcited by 46
  11. 2025A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applicationscited by 40
  12. 2024Assessment of Multimodal Large Language Models in Alignment with Human Valuescited by 36
38 more
  1. 2024A Robot Walks into a Bar: Can Language Models Serve as Creativity Support Tools for Comedy? An Evaluation of LLMs' Humour Alignment with Comedianscited by 29
  2. 2025From 1,000,000 Users to Every User: Scaling Up Personalized Preference for User-level Alignmentcited by 28
  3. 2024How do Large Language Models Navigate Conflicts between Honesty and Helpfulness?cited by 27
  4. 2025On Benchmarking Human-Like Intelligence in Machinescited by 24
  5. 2024Adaptive Retrieval-Augmented Generation for Conversational Systemscited by 22
  6. 2024Language Models in Dialogue: Conversational Maxims for Human-AI Interactionscited by 21
  7. 2024Mapping the Ethics of Generative AI: A Comprehensive Scoping Reviewcited by 19
  8. 2025Training language models to be warm and empathetic makes them less reliable and more sycophanticcited by 19
  9. 2024CEB: Compositional Evaluation Benchmark for Fairness in Large Language Modelscited by 17
  10. 2024Epistemic Injustice in Generative AIcited by 16
  11. 2023Robust Weight Signatures: Gaining Robustness as Easy as Patching Weights?cited by 13
  12. 2025Thinking beyond the anthropomorphic paradigm benefits LLM researchcited by 13
  13. 2025Generative Value Conflicts Reveal LLM Prioritiescited by 13
  14. 2023Can ChatGPT Perform Reasoning Using the IRAC Method in Analyzing Legal Scenarios Like a Lawyer?cited by 12
  15. 2025HumT DumT: Measuring and controlling human-like language in LLMscited by 11
  16. 2023ChatGPT in the Age of Generative AI and Large Language Models: A Concise Surveycited by 10
  17. 2025From tools to thieves: Measuring and understanding public perceptions of AI through crowdsourced metaphorscited by 9
  18. 2026Accommodation and Epistemic Vigilance: A Pragmatic Account of Why LLMs Fail to Challenge Harmful Beliefscited by 8
  19. 2023The Empty Signifier Problem: Towards Clearer Paradigms for Operationalising "Alignment" in Large Language Modelscited by 7
  20. 2024ChatGPT Role-play Dataset: Analysis of User Motives and Model Naturalnesscited by 7
  21. 2024Sociotechnical Implications of Generative Artificial Intelligence for Information Accesscited by 6
  22. 2024I Am Not Them: Fluid Identities and Persistent Out-group Bias in Large Language Modelscited by 5
  23. 2024CURATe: Benchmarking Personalised Alignment of Conversational AI Assistantscited by 5
  24. 2025CARE: Decoding Time Safety Alignment via Rollback and Introspection Interventioncited by 5
  25. 2025Normative Conflicts and Shallow AI Alignmentcited by 4
  26. 2023Artificial Intelligence and Aesthetic Judgmentcited by 3
  27. 2025Rethinking AI Cultural Alignmentcited by 3
  28. 2025Ontologies in Design: How Imagining a Tree Reveals Possibilites and Assumptions in Large Language Modelscited by 3
  29. 2025EVALUESTEER: Measuring Reward Model Steerability Towards Values and Preferencescited by 3
  30. 2026AI Alignment Breaks at the Edgecited by 3
  31. 2023Appropriateness is all you need!cited by 2
  32. 2026CALM: Culturally Self-Aware Language Modelscited by 2
  33. 2022Inclusive Artificial Intelligencecited by 1
  34. 2023ChatGPT, Large Language Technologies, and the Bumpy Road of Benefiting Humanitycited by 1
  35. 2023From Bytes to Biases: Investigating the Cultural Self-Perception of Large Language Modelscited by 1
  36. 2024On the Undecidability of Artificial Intelligence Alignment: Machines that Haltcited by 1
  37. 2024Ethics Whitepaper: Whitepaper on Ethical Research into Large Language Modelscited by 1
  38. 2025Conversational Alignment with Artificial Intelligence in Contextcited by 1

50 most cited of 61 All 61 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Should agentic conversational AI change how we think about ethics? Characterising an interactional ethics centred on respect
  2. 2025Gricean Norms as a Basis for Effective Collaboration
  3. 2023Align on the Fly: Adapting Chatbot Behavior to Established Norms
  4. 2024Towards Dialogues for Joint Human-AI Reasoning and Value Alignment
  5. 2026Toward a Theory of Value in AI Alignment
  6. 2024Contextual Moral Value Alignment Through Context-Based Aggregation
  7. 2026A Scoping Review of the Ethical Perspectives on Anthropomorphising Large Language Model-Based Conversational Agents
  8. 2024A Moral Imperative: The Need for Continual Superalignment of Large Language Models