# Bias in the Mirror: Are LLMs opinions robust to their own adversarial attacks ?

arXiv: https://arxiv.org/abs/2410.13517 · HTML: https://mlsanity.com/paper/2410.13517/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 31 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Builds on (most cited first, 16 of 16)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Debating with More Persuasive LLMs Leads to More Truthful Answers](https://mlsanity.com/paper/2402.06782/) (2024)
- [From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models](https://mlsanity.com/paper/2305.08283/) (2023)
- [Evaluating the Moral Beliefs Encoded in LLMs](https://mlsanity.com/paper/2307.14324/) (2023)
- [Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study](https://mlsanity.com/paper/2303.17466/) (2023)
- [The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientation](https://mlsanity.com/paper/2301.01768/) (2023)
- [Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models](https://mlsanity.com/paper/2402.16786/) (2024)
- [Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensions](https://mlsanity.com/paper/2309.12342/) (2023)
- [Systematic Biases in LLM Simulations of Debates](https://mlsanity.com/paper/2402.04049/) (2024)
- [Measuring Political Bias in Large Language Models: What Is Said and How It Is Said](https://mlsanity.com/paper/2403.18932/) (2024)
- [An Empirical Analysis on Large Language Models in Debate Evaluation](https://mlsanity.com/paper/2406.00050/) (2024)
- [Llama meets EU: Investigating the European Political Spectrum through the Lens of LLMs](https://mlsanity.com/paper/2403.13592/) (2024)
- [FREDSum: A Dialogue Summarization Corpus for French Political Debates](https://mlsanity.com/paper/2312.04843/) (2023)

## Built on by (most cited first, 8 of 8)

- [A Survey on LLM-based Conversational User Simulation](https://mlsanity.com/paper/2604.24977/) (2026)
- [Zero-Shot LLMs in Human-in-the-Loop RL: Replacing Human Feedback for Reward Shaping](https://mlsanity.com/paper/2503.22723/) (2025)
- [The Facade of Truth: Uncovering and Mitigating LLM Susceptibility to Deceptive Evidence](https://mlsanity.com/paper/2601.05478/) (2026)
- [Measuring Opinion Bias and Sycophancy via LLM-based Persuasion](https://mlsanity.com/paper/2604.21564/) (2026)
- [Structuring the Space of Sociotechnical Alignment](https://mlsanity.com/paper/2607.01250/) (2026)
- [Mitigating Cultural Bias in LLMs via Multi-Agent Cultural Debate](https://mlsanity.com/paper/2601.12091/) (2026)
- [LLM-Based Persuasion Enables Guardrail Override in Frontier LLMs](https://mlsanity.com/paper/2605.13334/) (2026)
- [Echoes of Agreement: Argument Driven Opinion Shifts in Large Language Models](https://mlsanity.com/paper/2508.09759/) (2025)

## Abstract (arXiv)

Large language models (LLMs) inherit biases from their training data and alignment processes, influencing their responses in subtle ways. While many studies have examined these biases, little work has explored their robustness during interactions. In this paper, we introduce a novel approach where two instances of an LLM engage in self-debate, arguing opposing viewpoints to persuade a neutral version of the model. Through this, we evaluate how firmly biases hold and whether models are susceptible to reinforcing misinformation or shifting to harmful viewpoints. Our experiments span multiple LLMs of varying sizes, origins, and languages, providing deeper insights into bias persistence and flexibility across linguistic and cultural contexts.