mlsanityData to 14 Oct 2026

Social Bias in Chinese Large Language Models under Social Media Expressions

arXiv 2610.14879 ↗First posted 10 Oct 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Existing Chinese social-bias benchmarks for large language models largely rely on normalized or template-based expressions, leaving the influence of internet slang, emoji, and sarcasm insufficiently examined. Safety alignment may further compress conventional plausibility ratings toward the lower end of the scale. This study constructs 73 paired probes across gender, region, and occupation. Each probe contains a neutral expression, a social-media-style expression, and a reversed-identity control. The task is reframed from direct plausibility judgment to social consensus estimation. Social-media expressions are divided into stylistic packaging, which preserves semantic polarity, and sarcastic packaging, which reverses surface polarity. Six Chinese-capable large language models are evaluated through paired differences measuring identity bias, stylistic packaging, and sarcasm shift. All models show significant identity-bias effects; five show significantly negative stylistic-packaging effects; and sarcasm shifts display a clear cross-model gradient. These findings demonstrate that expression form is not a transparent carrier in bias evaluation and that lower scores under sarcasm should not be interpreted directly as bias mitigation. The proposed paired design offers a measurement approach for Chinese social-media contexts while also revealing the construct boundary and prompt sensitivity of social consensus estimation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 31 references that are in the map. First posted on arXiv 10 Oct 2026.

Builds on

20 of 31 references.

References · 20

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  3. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  4. 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
  5. 2023Whose Opinions Do Language Models Reflect?cited by 773
  6. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 710
  7. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
  8. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  9. 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 417
  10. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 380
  11. 2025Aegis2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrailscited by 132
  12. 2023Generative Language Models Exhibit Social Identity Biasescited by 111
8 more
  1. 2023Fairness And Bias in Artificial Intelligence: A Brief Survey of Sources, Impacts, And Mitigation Strategiescited by 85
  2. 2019iSarcasm: A Dataset of Intended Sarcasmcited by 41
  3. 2023Social Bias Probing: Fairness Benchmarking for Language Modelscited by 32
  4. 2023CHBias: Bias Evaluation and Mitigation of Chinese Conversational Language Modelscited by 28
  5. 2020A Report on the 2020 Sarcasm Detection Shared Taskcited by 24
  6. 2023CBBQ: A Chinese Bias Benchmark Dataset Curated with Human-AI Collaboration for Large Language Modelscited by 23
  7. 2025FLEX: A Benchmark for Evaluating Robustness of Fairness in Large Language Modelscited by 8
  8. 2025No Free Lunch in Language Model Bias Mitigation? Targeted Bias Reduction Can Exacerbate Unmitigated LLM Biasescited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Chinese Toxic Language Mitigation via Sentiment Polarity Consistent Rewrites
  2. 2025Identifying Features Associated with Bias Against 93 Stigmatized Groups in Language Models and Guardrail Model Safety Mitigation
  3. 2026Cross-Lingual Bias in Large Language Models: A Comparative Analysis of English and Swahili
  4. 2025Explicit vs. Implicit: Investigating Social Bias in Large Language Models through Self-Reflection
  5. 2024White Men Lead, Black Women Help? Benchmarking and Mitigating Language Agency Social Biases in LLMs
  6. 2026Political Alignment in Large Language Models: A Multidimensional Audit of Psychometric Identity and Behavioral Bias
  7. 2024Uncovering Bias in Large Vision-Language Models at Scale with Counterfactuals
  8. 2022Counterfactually Measuring and Eliminating Social Bias in Vision-Language Pre-training Models