mlsanityData to 14 Oct 2026

Adversarial Alignment: Ensuring Value Consistency in Large Language Models for Sensitive Domains

arXiv 2601.13137 ↗First posted 19 Jan 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

With the wide application of large language models (LLMs), the problems of bias and value inconsistency in sensitive domains have gradually emerged, especially in terms of race, society and politics. In this paper, we propose an adversarial alignment framework, which enhances the value consistency of the model in sensitive domains through continued pre-training, instruction fine-tuning and adversarial training. In adversarial training, we use the Attacker to generate controversial queries, the Actor to generate responses with value consistency, and the Critic to filter and ensure response quality. Furthermore, we train a Value-Consistent Large Language Model, VC-LLM, for sensitive domains, and construct a bilingual evaluation dataset in Chinese and English. The experimental results show that VC-LLM performs better than the existing mainstream models in both Chinese and English tests, verifying the effectiveness of the method. Warning: This paper contains examples of LLMs that are offensive or harmful in nature.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 43 references that are in the map. First posted on arXiv 19 Jan 2026.

Builds on

33 of 43 references.

References · 33

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2024Qwen2.5 Technical Reportcited by 4,942
  4. 2024GPT-4o System Cardcited by 4,588
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  6. 2023Mistral 7Bcited by 3,164
  7. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  8. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  9. 2024Yi: Open Foundation Models by 01.AIcited by 719
  10. 2023Baichuan 2: Open Large-scale Language Modelscited by 670
  11. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 656
  12. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
21 more
  1. 2024InternLM2 Technical Reportcited by 397
  2. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 377
  3. 2024Alignment faking in large language modelscited by 360
  4. 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 305
  5. 2024Measuring Political Bias in Large Language Models: What Is Said and How It Is Saidcited by 89
  6. 2021Mitigating Political Bias in Language Models Through Reinforced Calibrationcited by 73
  7. 2023Ethical Reasoning over Moral Alignment: A Case and Framework for In-Context Ethical Policies in LLMscited by 64
  8. 2024Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactionscited by 51
  9. 2024Moral Alignment for LLM Agentscited by 39
  10. 2023Rethinking Machine Ethics -- Can LLMs Perform Moral Reasoning through the Lens of Moral Theories?cited by 36
  11. 2024MAP: Multi-Human-Value Alignment Palettecited by 20
  12. 2024Mapping and Influencing the Political Ideology of Large Language Models using Synthetic Personascited by 16
  13. 2023Ziya2: Data-centric Learning is All LLMs Needcited by 15
  14. 2023TigerBot: An Open Multilingual Multitask LLMcited by 15
  15. 2024Bias and Volatility: A Statistical Framework for Evaluating Large Language Model's Stereotypes and the Associated Generation Inconsistencycited by 15
  16. 2024PoliTune: Analyzing the Impact of Data Selection and Fine-Tuning on Economic and Political Biases in Large Language Modelscited by 11
  17. 2024Examining the Influence of Political Bias on Large Language Model Performance in Stance Classificationcited by 9
  18. 2025WenyanGPT: A Large Language Model for Classical Chinese Taskscited by 6
  19. 2024Revealing Fine-Grained Values and Opinions in Large Language Modelscited by 5
  20. 2024Measuring Spiritual Values and Bias of Large Language Modelscited by 1
  21. 2025Are LLMs (Really) Ideological? An IRT-based Analysis and Alignment Tool for Perceived Socio-Economic Bias in LLMscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Flames: Benchmarking Value Alignment of LLMs in Chinese
  2. 2023Large Language Model Alignment: A Survey
  3. 2023CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibility
  4. 2024Can Reinforcement Learning Unlock the Hidden Dangers in Aligned Large Language Models?
  5. 2025Benchmarking Adversarial Robustness to Bias Elicitation in Large Language Models: Scalable Automated Assessment with LLM-as-a-Judge
  6. 2025Revealing the Intrinsic Ethical Vulnerability of Aligned Large Language Models
  7. 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges
  8. 2025Internal Value Alignment in Large Language Models through Controlled Value Vector Activation