mlsanityData to 7 Oct 2026

Should We Respect LLMs? A Cross-Lingual Study on the Influence of Prompt Politeness on LLM Performance

arXiv 2402.14531 ↗First posted 22 Feb 2024cs.CLcited by 41

Not on the map. Why?

Abstract (arXiv)

We investigate the impact of politeness levels in prompts on the performance of large language models (LLMs). Polite language in human communications often garners more compliance and effectiveness, while rudeness can cause aversion, impacting response quality. We consider that LLMs mirror human communication traits, suggesting they align with human cultural norms. We assess the impact of politeness in prompts on LLMs across English, Chinese, and Japanese tasks. We observed that impolite prompts often result in poor performance, but overly polite language does not guarantee better outcomes. The best politeness level is different according to the language. This phenomenon suggests that LLMs not only reflect human behavior but are also influenced by language, particularly in different cultural contexts. Our findings highlight the need to factor in politeness for cross-cultural natural language processing and LLM usage.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Builds on 18 of its 35 references that are in the map. First posted on arXiv 22 Feb 2024.

Builds on

18 of 35 references.

References · 18

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  4. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  5. 2017Deep reinforcement learning from human preferencescited by 3,846
  6. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  7. 2015Teaching Machines to Read and Comprehendcited by 1,039
  8. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  9. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  10. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  11. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 612
  12. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
6 more
  1. 2021XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languagescited by 346
  2. 2020CLUE: A Chinese Language Understanding Evaluation Benchmarkcited by 277
  3. 2023Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Studycited by 203
  4. 2022Gender Bias in Masked Language Models for Multiple Languagescited by 60
  5. 2023CHBias: Bias Evaluation and Mitigation of Chinese Conversational Language Modelscited by 25
  6. 2023Gender Bias in BERT -- Measuring and Analysing Biases through Sentiment Rating in a Realistic Downstream Classification Taskcited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 18; 2026: 16.

41 papers.

Sort:

By area:

  1. 2024Survey of Cultural Awareness in Language Models: Text and Beyondcited by 101
  2. 2025MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluationcited by 73
  3. 2024Social Bias Evaluation for Large Language Models Requires Prompt Variationscited by 39
  4. 2025Does Tone Change the Answer? Evaluating Prompt Politeness Effects on Modern LLMs: GPT, Gemini, and LLaMAcited by 23
  5. 2025Firm or Fickle? Evaluating Large Language Models Consistency in Sequential Interactionscited by 12
  6. 2025Out of Style: RAG's Fragility to Linguistic Variationcited by 10
  7. 2025Prompting Science Report 1: Prompt Engineering is Complicated and Contingentcited by 8
  8. 2025Expect the Unexpected: FailSafe Long Context QA for Financecited by 7
  9. 2025Mind Your Tone: Investigating How Prompt Politeness Affects LLM Accuracy (short paper)cited by 7
  10. 2025Interactional Fairness in LLM Multi-Agent Systems: An Evaluation Frameworkcited by 6
  11. 2024Minstrel: Structural Prompt Generation with Multi-Agents Coordination for Non-AI Expertscited by 4
  12. 2025Instability in Downstream Task Performance During LLM Pretrainingcited by 4
29 more
  1. 2024The Struggles of LLMs in Cross-lingual Code Clone Detectioncited by 3
  2. 2024Development and bilingual evaluation of Japanese medical large language model within reasonably low computational resourcescited by 3
  3. 2025Traveling Across Languages: Benchmarking Cross-Lingual Consistency in Multimodal LLMscited by 3
  4. 2025Humans overrely on overconfident language models, across languagescited by 3
  5. 2024M2M-Gen: A Multimodal Framework for Automated Background Music Generation in Japanese Manga Using Large Language Modelscited by 2
  6. 2024ActionCOMET: A Zero-shot Approach to Learn Image-specific Commonsense Concepts about Actionscited by 2
  7. 2025Use Me Wisely: AI-Driven Assessment for LLM Prompting Skills Developmentcited by 2
  8. 2025A Japanese Language Model and Three New Evaluation Benchmarks for Pharmaceutical NLPcited by 2
  9. 2025Learning a Single Token to Replace Long System Prompts in LLMscited by 2
  10. 2026Chasing the Public Score: User Pressure and Evaluation Exploitation in Coding Agent Workflowscited by 2
  11. 2025CIE: Controlling Language Model Text Generations Using Continuous Signalscited by 1
  12. 2026Small Talk, Big Impact: The Energy Cost of Thanking AIcited by 1
  13. 2026Caring Without Feeling: Affective Dynamics as the Control Layer of Human-AI Agent Collaborationcited by 1
  14. 2025Effectiveness of Zero-shot-CoT in Japanese Promptscited by 0
  15. 2025Pragmatics beyond humans: meaning, communication, and LLMscited by 0
  16. 2025LLMs for LLMs: A Structured Prompting Methodology for Long Legal Documentscited by 0
  17. 2026ADAB: Arabic Dataset for Automated Politeness Benchmarking -- A Large-Scale Resource for Computational Sociopragmaticscited by 0
  18. 2026Imperative Interference: Social Register Shapes Instruction Topology in Large Language Modelscited by 0
  19. 2026The Role of Emotional Stimuli and Intensity in Shaping Large Language Model Behaviorcited by 0
  20. 2026No Universal Courtesy: A Cross-Linguistic, Multi-Model Study of Politeness Effects on LLMs Using the PLUM Corpuscited by 0
  21. 2026Visual Fingerprints for LLM Generation Comparisoncited by 0
  22. 2026Phoenix-VL 1.5 Medium Technical Reportcited by 0
  23. 2026Mind Your Tone: Does Tone Alter LLM Performance?cited by 0
  24. 2026The Role of Prompt Language and Translation-Theory-Driven Prompts in Large Language Models: A Case Study on Spanish-Chinese Journalistic Translationcited by 0
  25. 2026Do Influence Tactics Matter? Investigating Prompt Framing Effects in LLM Code Generationcited by 0
  26. 2026Talking Past the Machine: Morality, Politeness, and Alignment in Human-AI Dialoguecited by 0
  27. 2026Framing the Narrative: Ideological Mimicry in Large Language Modelscited by 0
  28. 2026Evaluating the Robustness of Japanese LLMs to IME-Related and Typographical Errorscited by 0
  29. 2026Generalization Is Stability, Not Accuracy: Multi-Axis Evaluation of LLMscited by 0