mlsanityData to 14 Oct 2026

FairEval: Evaluating Fairness in LLM-Based Recommendations with Personality Awareness

arXiv 2504.07801 ↗First posted 10 Apr 2025cs.IR cs.AI cs.HCcited by 8

Not on the map. Why?

Abstract (arXiv)

Recent advances in Large Language Models (LLMs) have enabled their application to recommender systems (RecLLMs), yet concerns remain regarding fairness across demographic and psychological user dimensions. We introduce FairEval, a novel evaluation framework to systematically assess fairness in LLM-based recommendations. FairEval integrates personality traits with eight sensitive demographic attributes,including gender, race, and age, enabling a comprehensive assessment of user-level bias. We evaluate models, including ChatGPT 4o and Gemini 1.5 Flash, on music and movie recommendations. FairEval's fairness metric, PAFS, achieves scores up to 0.9969 for ChatGPT 4o and 0.9997 for Gemini 1.5 Flash, with disparities reaching 34.79 percent. These results highlight the importance of robustness in prompt sensitivity and support more inclusive recommendation systems.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 28 of its 65 references that are in the map. First posted on arXiv 10 Apr 2025.

Builds on

28 of 65 references.

References · 28

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  3. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 619
  4. 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 296
  5. 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 226
  6. 2023Recommender Systems in the Era of Large Language Models (LLMs)cited by 189
  7. 2021Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasetscited by 181
  8. 2019Fairness in Recommendation Ranking through Pairwise Comparisonscited by 127
  9. 2018Equity of Attention: Amortizing Individual Fairness in Rankingscited by 124
  10. 2023Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendationcited by 114
  11. 2024Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Eracited by 98
  12. 2017Beyond Parity: Fairness Objectives for Collaborative Filteringcited by 93
16 more
  1. 2019Managing Popularity Bias in Recommender Systems with Personalized Re-rankingcited by 82
  2. 2017Multisided Fairness for Recommendationcited by 70
  3. 2024Thinking Fair and Slow: On the Efficacy of Structured Prompts for Debiasing Language Modelscited by 45
  4. 2024Large Language Models as Recommender Systems: A Study of Popularity Biascited by 26
  5. 2023Sensitivity and Robustness of Large Language Models to Prompt Template in Japanese Text Classification Taskscited by 25
  6. 2022Joint Multisided Exposure Fairness for Recommendationcited by 22
  7. 2023Fairness and Diversity in Recommender Systems: A Surveycited by 22
  8. 2021A Survey on Personality-Aware Recommendation Systemscited by 18
  9. 2021A Framework for Fairness: A Systematic Review of Existing Fair AI Solutionscited by 18
  10. 2023Fairness of ChatGPT and the Role Of Explainable-Guided Promptscited by 13
  11. 2023Large Language Models are Not Stable Recommender Systemscited by 13
  12. 2020Opportunistic Multi-aspect Fairness through Personalized Re-rankingcited by 11
  13. 2016Personality Traits and Echo Chambers on Facebookcited by 9
  14. 2023Dynamic fairness-aware recommendation through multi-agent social choicecited by 9
  15. 2024ABCFair: an Adaptable Benchmark approach for Comparing Fairness Methodscited by 8
  16. 2024Unveiling Bias in Fairness Evaluations of Large Language Models: A Critical Literature Review of Music and Movie Recommendation Systemscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 6.

8 papers.

  1. 2026Conv-FinRe: A Conversational and Longitudinal Benchmark for Utility-Grounded Financial Recommendationcited by 4
  2. 2025PerFairX: Is There a Balance Between Fairness and Personality in Large Language Model Recommendations?cited by 3
  3. 2026IndicSafe: A Benchmark for Evaluating Multilingual LLM Safety in South Asiacited by 3
  4. 2026Ads in AI Chatbots? An Analysis of How Large Language Models Navigate Conflicts of Interestcited by 3
  5. 2026Emergence of Biased Consensus in Multi-Agent LLM Debatescited by 3
  6. 2026Polarization by Default: Auditing Recommendation Bias in LLM-Based Content Curationcited by 1
  7. 2025PBiLoss: Popularity-Aware Regularization to Improve Fairness in Graph-Based Recommender Systemscited by 0
  8. 2026Rethinking Fairness in LLM-Based Recommender Systems: A Surveycited by 0