mlsanityData to 14 Oct 2026

Talking Past the Machine: Morality, Politeness, and Alignment in Human-AI Dialogue

arXiv 2609.21401 ↗First posted 18 Sep 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Conversational AI systems produce fluent, socially appropriate responses, yet whether they participate in cooperative communication or merely simulate its surface forms remains unclear - a question central to how these systems are evaluated, trusted, and designed. This study investigates how morality, politeness, and alignment - three dimensions central to cooperative dialogue - function in human-AI interaction compared to human-human conversation. We analyze 15,881 human-ChatGPT and 10,784 human-human multi-turn dialogues, using mixed-effects models to identify which features predict turn-to-turn alignment. We observe a consistent dissociation: AI produces the surface features of cooperative communication without the underlying social architecture. Moral output appears preconfigured rather than negotiated; warmth is generated without face sensitivity; linguistic convergence declines persistently. Most strikingly, the cooperative mechanisms themselves reverse direction: hedging and softening associated with greater accommodation between humans are associated with reduced alignment when produced by AI, and purity framing associated with human divergence coincides with users converging toward the AI. Agency - giving users room to shape the exchange - is the most consistent predictor of alignment across both interaction types, while lower moral assertiveness in more recent models is not accompanied by better cooperation. Together these patterns suggest that AI reproduces the surface of cooperation without the mutual adaptation that grounds it between humans - and, more surprisingly, that mechanisms sustaining human accommodation can run in reverse with AI, suggesting a turn-level view may be insufficient for interaction-level success.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 45 references that are in the map. First posted on arXiv 18 Sep 2026.

Builds on

17 of 45 references.

References · 17

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
  4. 2023Whose Opinions Do Language Models Reflect?cited by 773
  5. 2020Aligning AI With Shared Human Valuescited by 723
  6. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
  7. 2024MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialoguescited by 250
  8. 2023Evaluating the Moral Beliefs Encoded in LLMscited by 215
  9. 2021Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Docited by 178
  10. 2023Moral Foundations of Large Language Modelscited by 85
  11. 2023Grounding Gaps in Language Model Generationscited by 47
  12. 2024Should We Respect LLMs? A Cross-Lingual Study on the Influence of Prompt Politeness on LLM Performancecited by 41
5 more
  1. 2025Interaction Context Often Increases Sycophancy in LLMscited by 26
  2. 2024Attributions toward Artificial Agents in a modified Moral Turing Testcited by 18
  3. 2025MetaMind: Modeling Human Social Thoughts with Metacognitive Multi-Agent Systemscited by 13
  4. 2026LLMs and people both learn to form conventions -- just not with each othercited by 5
  5. 2026Accommodation Goes Both Ways: Studying Linguistic Convergence Between Humans and Language Modelscited by 2

Built on by

Not cited yet.