# Critical Acclaim Orientation in Large Language Models: Evidence from Film Preference Elicitation

arXiv: https://arxiv.org/abs/2608.06955 · HTML: https://mlsanity.com/paper/2608.06955/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 42 references that are in the map.
- First posted on arXiv 7 Aug 2026.

## Builds on (most cited first, 11 of 11)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendation](https://mlsanity.com/paper/2305.07609/) (2023)
- [The Geometry of Culture: Analyzing Meaning through Word Embeddings](https://mlsanity.com/paper/1803.09288/) (2018)
- [Randomness, Not Representation: The Unreliability of Evaluating Cultural Alignment in LLMs](https://mlsanity.com/paper/2503.08688/) (2025)
- [What's in a Name? Auditing Large Language Models for Race and Gender Bias](https://mlsanity.com/paper/2402.14875/) (2024)
- [Normative Evaluation of Large Language Models with Everyday Moral Dilemmas](https://mlsanity.com/paper/2501.18081/) (2025)
- [How Deep Is Representational Bias in LLMs? The Cases of Caste and Religion](https://mlsanity.com/paper/2508.03712/) (2025)
- [Trustworthy Social Bias Measurement](https://mlsanity.com/paper/2212.11672/) (2022)
- [Revealing Potential Biases in LLM-Based Recommender Systems in the Cold Start Setting](https://mlsanity.com/paper/2508.20401/) (2025)
- [Understanding the Interplay of Scale, Data, and Bias in Language Models: A Case Study with BERT](https://mlsanity.com/paper/2407.21058/) (2024)

## Abstract (arXiv)

Large language models (LLMs) are trained on corpora that contain expressions of human judgment about films, books, music, and more. Yet whether LLMs systematically reproduce evaluative hierarchies remains unclear. Prior research on cultural bias in LLMs suggests competing expectations: models may mirror the popularity signals of internet texts, or may reproduce forms of prestige embedded in critical discourse. We probe this question through a study of film evaluations with eight models from four families (Anthropic, OpenAI, Alibaba, and Mistral), using a 200-film benchmark partitioned into critically acclaimed, commercially successful, and dual-legitimacy (critical acclaim + commercial success) films. Across 20,000 pairwise forced-choice comparisons per model analyzed with Bradley--Terry estimation, we observe a consistent critical acclaim orientation with all models: critically acclaimed yet commercially obscure films are selected over commercially successful yet critically unrecognized ones. This pattern grows with model scale within each family. In addition, nested OLS regression analyses show that evaluative orientation, public visibility, and popular reception distinctly help explain preferences. Adjusting for public visibility reverses the models' preference for dual-legitimacy films over critical acclaim-only films, while additionally accounting for popular reception attenuates much of the disadvantage of films with commercial success only. Finally, evaluative and recommendation-oriented prompt framings produce divergent rankings, suggesting that critical acclaim orientation may manifest indirectly in real-world LLM deployments.