# Taming the Long Tail: Efficient Item-wise Sharpness-Aware Minimization for LLM-based Recommender Systems

arXiv: https://arxiv.org/abs/2603.12752 · HTML: https://mlsanity.com/paper/2603.12752/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 71 references that are in the map.
- First posted on arXiv 13 Mar 2026.

## Builds on (most cited first, 27 of 27)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Sharpness-Aware Minimization for Efficiently Improving Generalization](https://mlsanity.com/paper/2010.01412/) (2020)
- [TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation](https://mlsanity.com/paper/2305.00447/) (2023)
- [Towards Open-World Recommendation with Knowledge Augmentation from Large Language Models](https://mlsanity.com/paper/2306.10933/) (2023)
- [A Bi-Step Grounding Paradigm for Large Language Models in Recommendation Systems](https://mlsanity.com/paper/2308.08434/) (2023)
- [MACRec: a Multi-Agent Collaboration Framework for Recommendation](https://mlsanity.com/paper/2402.15235/) (2024)
- [Knowledge Graph Retrieval-Augmented Generation for LLM-based Recommendation](https://mlsanity.com/paper/2501.02226/) (2025)
- [LLaRA: Large Language-Recommendation Assistant](https://mlsanity.com/paper/2312.02445/) (2023)
- [LLM-ESR: Large Language Models Enhancement for Long-tailed Sequential Recommendation](https://mlsanity.com/paper/2405.20646/) (2024)
- [CoRAL: Collaborative Retrieval-Augmented Large Language Models Improve Long-tail Recommendation](https://mlsanity.com/paper/2403.06447/) (2024)
- [A Model of Two Tales: Dual Transfer Learning Framework for Improved Long-tail Item Recommendation](https://mlsanity.com/paper/2010.15982/) (2020)
- [Self-supervised Geometric Features Discovery via Interpretable Attention for Vehicle Re-Identification and Beyond](https://mlsanity.com/paper/2010.09221/) (2020)
- [Swift Sampler: Efficient Learning of Sampler by 10 Parameters](https://mlsanity.com/paper/2410.05578/) (2024)
- [Differential Private Knowledge Transfer for Privacy-Preserving Cross-Domain Recommendation](https://mlsanity.com/paper/2202.04893/) (2022)
- [Item-side Fairness of Large Language Model-based Recommendation System](https://mlsanity.com/paper/2402.15215/) (2024)
- [Language Representations Can be What Recommenders Need: Findings and Potentials](https://mlsanity.com/paper/2407.05441/) (2024)
- [SPRec: Self-Play to Debias LLM-based Recommendation](https://mlsanity.com/paper/2412.09243/) (2024)
- [YOLO-PPA based Efficient Traffic Sign Detection for Cruise Control in Autonomous Driving](https://mlsanity.com/paper/2409.03320/) (2024)
- [Empowering Long-tail Item Recommendation through Cross Decoupling Network (CDN)](https://mlsanity.com/paper/2210.14309/) (2022)
- [From Points to Coalitions: Hierarchical Contrastive Shapley Values for Prioritizing Data Samples](https://mlsanity.com/paper/2512.19363/) (2025)
- [CEC-Zero: Zero-Supervision Character Error Correction with Self-Generated Rewards](https://mlsanity.com/paper/2512.23971/) (2025)
- [CompTrack: Information Bottleneck-Guided Low-Rank Dynamic Token Compression for Point Cloud Tracking](https://mlsanity.com/paper/2511.15580/) (2025)
- [CITIES: Contextual Inference of Tail-Item Embeddings for Sequential Recommendation](https://mlsanity.com/paper/2105.10868/) (2021)
- [Intersectional Two-sided Fairness in Recommendation](https://mlsanity.com/paper/2402.02816/) (2024)
- [Contextual Bandits for Unbounded Context Distributions](https://mlsanity.com/paper/2408.09655/) (2024)
- [Sharpness-Aware Graph Collaborative Filtering](https://mlsanity.com/paper/2307.08910/) (2023)

## Abstract (arXiv)

Large Language Model-based Recommender Systems (LRSs) have recently emerged as a new paradigm in sequential recommendation by directly adopting LLMs as backbones. While LRSs demonstrate strong knowledge utilization and instruction-following abilities, they have not been systematically studied under the long-standing long-tail problem. In this paper, we conduct an empirical study and reveal that LRSs face two distinct types of long-tail: i) prior long-tail, inherited implicitly from pretraining corpora, and ii) data long-tail, originating from skewed recommendation datasets. Our analysis shows that both contribute to the performance disparity between head and tail items, with the intersection of the two heads exhibiting an even stronger head effect. Nevertheless, the overall performance distribution in LRSs, especially on the tail, remains dominated by the data long-tail. To address this challenge, we propose Efficient Item-wise Sharpness-Aware Minimization (EISAM), a novel optimization framework that improves tail-item performance by adaptively regularizing the loss landscape at the item level. EISAM introduces an efficient penalty design that captures fine-grained item-specific sharpness while maintaining computational scalability for LLMs. In addition, we derive a generalization bound for EISAM. Our theoretical analysis shows that the bound decreases at a faster rate under our item-wise regularization, offering theoretical support for its effectiveness. Extensive experiments on three real-world datasets demonstrate that EISAM significantly boosts tail-item recommendation performance while preserving overall quality, establishing the first systematic solution to the long-tail problem in LRSs.