mlsanityData to 14 Oct 2026

Extending Beacon to Hindi: Cultural Adaptation Drives Cross-Lingual Sycophancy

arXiv 2602.00046 ↗First posted 19 Jan 2026cs.LG cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Sycophancy, the tendency of language models to prioritize agreement with user preferences over principled reasoning, has been identified as a persistent alignment failure in English-language evaluations. However, it remains unclear whether such diagnostics generalize across languages and cultural contexts. We extend the Beacon single-turn forced-choice sycophancy diagnostic to Hindi through a controlled three-condition design: English original, Hindi literal translation, and Hindi culturally adapted prompts. We evaluate four open-weight instruction-tuned models on 50 prompts per condition, enabling separation of language encoding effects from cultural adaptation effects. Across all models, sycophancy rates are consistently higher for culturally adapted Hindi prompts than for English, with absolute differences ranging from 12.0 to 16.0 percentage points. A decomposition on Qwen 2.5-Coder-7B shows that cultural adaptation (delta = 14.0%, 95% CI: [4.0%, 26.0%]) accounts for the majority of this gap, while language encoding contributes minimally (delta = 2.0%, 95% CI: [0.0%, 6.0%]). Category-level analysis reveals that advice prompts exhibit the largest cross-lingual differences (20-25 percentage points), achieving statistical significance in two of four models. These findings indicate that alignment behaviors measured in English may not transfer uniformly across languages and that culturally grounded prompt framing plays a substantial role. We release all datasets and evaluation code to support replication and extension.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 4 of its 6 references that are in the map. First posted on arXiv 19 Jan 2026.

Builds on

4 of 6 references.

References · 4

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  3. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  4. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 742

Built on by

Not cited yet.