# Does LLM Focus on the Right Words? Mitigating Context Bias in LLM-based Recommenders

arXiv: https://arxiv.org/abs/2510.10978 · HTML: https://mlsanity.com/paper/2510.10978/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in LLM reasoning for recommendation, one of 17 directions in LLM reasoning.
- First posted on arXiv 13 Oct 2025.

## Placed in

- LLM reasoning / [LLM reasoning for recommendation](https://mlsanity.com/lr/llm-reasoning-for-recommendation/)

## Built on by (most cited first, 2 of 2)

- [The Pitfall of Scaling Up: Uncovering and Mitigating Popularity Bias Amplification in Scaling Transformer-based Recommenders](https://mlsanity.com/paper/2606.21911/) (2026)
- [LBR: Towards Mitigating Length Bias in Large Language Models for Recommendation](https://mlsanity.com/paper/2607.04270/) (2026)

## Abstract (arXiv)

Large language models (LLMs), owing to their extensive open-domain knowledge and semantic reasoning capabilities, have been increasingly integrated into recommender systems (RS). However, a substantial gap remains between the pre-training objectives of LLMs and the specific requirements of recommendation tasks. To address this gap, supervised fine-tuning (SFT) is commonly performed on specially curated recommendation datasets to further enhance their predictive ability. Despite its success, SFT exhibits a critical limitation: it induces Context Bias, whereby the model over-relies on auxiliary tokens, such as task descriptions and prefix-generated tokens, while underutilizing core user interaction tokens that encode user-specific preferences. This bias not only undermines recommendation accuracy but also raises unfairness concerns. To address this issue, we propose Group Distributionally Robust Optimization-based Tuning (GDRT), a novel fine-tuning paradigm that enforces consistent model performance across token groups with varying degrees of relevance to auxiliary tokens. By adaptively upweighting underperforming groups, typically those weakly correlated with auxiliary tokens, GDRT shifts the model's attention from superficial auxiliary cues to informative user interaction tokens, thereby mitigating context bias. Extensive experiments conducted on three public datasets demonstrate that GDRT effectively mitigates context bias, yielding substantial improvements in recommendation accuracy (with an average NDCG@10 gain of 24.29%) and significantly enhancing recommendation fairness. The code is available at https://github.com/WANGBohaO-jpg/GDRT.