# Smoothie-Qwen: Post-Hoc Smoothing to Reduce Language Bias in Multilingual LLMs

arXiv: https://arxiv.org/abs/2507.05686 · HTML: https://mlsanity.com/paper/2507.05686/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 12 references that are in the map.
- First posted on arXiv 8 Jul 2025.

## Builds on (most cited first, 10 of 10)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies](https://mlsanity.com/paper/2308.03188/) (2023)
- [KMMLU: Measuring Massive Multitask Language Understanding in Korean](https://mlsanity.com/paper/2402.11548/) (2024)
- [Understanding and Mitigating Language Confusion in LLMs](https://mlsanity.com/paper/2406.20052/) (2024)
- [Self-Debiasing Large Language Models: Zero-Shot Recognition and Reduction of Stereotypes](https://mlsanity.com/paper/2402.01981/) (2024)
- [Harnessing Large Language Models as Post-hoc Correctors](https://mlsanity.com/paper/2402.13414/) (2024)
- [BanStereoSet: A Dataset to Measure Stereotypical Social Biases in LLMs for Bangla](https://mlsanity.com/paper/2409.11638/) (2024)

## Built on by (most cited first, 2 of 2)

- [Making Qwen3 Think in Korean with Reinforcement Learning](https://mlsanity.com/paper/2508.10355/) (2025)
- [Language Confusion Gate: Language-Aware Decoding Through Model Self-Distillation](https://mlsanity.com/paper/2510.17555/) (2025)

## Abstract (arXiv)

Multilingual large language models (LLMs) often exhibit language confusion, a tendency to generate responses in a dominant language irrespective of the prompt's language. To address this, we propose Smoothie-Qwen, a lightweight, post-hoc method that mitigates language bias without retraining. This technique selectively adjusts token-level output probabilities to effectively suppress undesired language generation. Applied to the Qwen model, our method reduces unintended Chinese output by over 95% while preserving task accuracy on multilingual benchmarks. This work provides a practical and efficient solution for enhancing the language controllability of LLMs, making them more reliable for global applications.