mlsanityData to 14 Oct 2026

Critical Acclaim Orientation in Large Language Models: Evidence from Film Preference Elicitation

arXiv 2608.06955 ↗First posted 7 Aug 2026cs.AI cs.CYcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are trained on corpora that contain expressions of human judgment about films, books, music, and more. Yet whether LLMs systematically reproduce evaluative hierarchies remains unclear. Prior research on cultural bias in LLMs suggests competing expectations: models may mirror the popularity signals of internet texts, or may reproduce forms of prestige embedded in critical discourse. We probe this question through a study of film evaluations with eight models from four families (Anthropic, OpenAI, Alibaba, and Mistral), using a 200-film benchmark partitioned into critically acclaimed, commercially successful, and dual-legitimacy (critical acclaim + commercial success) films. Across 20,000 pairwise forced-choice comparisons per model analyzed with Bradley--Terry estimation, we observe a consistent critical acclaim orientation with all models: critically acclaimed yet commercially obscure films are selected over commercially successful yet critically unrecognized ones. This pattern grows with model scale within each family. In addition, nested OLS regression analyses show that evaluative orientation, public visibility, and popular reception distinctly help explain preferences. Adjusting for public visibility reverses the models' preference for dual-legitimacy films over critical acclaim-only films, while additionally accounting for popular reception attenuates much of the disadvantage of films with commercial success only. Finally, evaluative and recommendation-oriented prompt framings produce divergent rankings, suggesting that critical acclaim orientation may manifest indirectly in real-world LLM deployments.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 11 of its 42 references that are in the map. First posted on arXiv 7 Aug 2026.

Builds on

11 of 42 references.

References · 11

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  3. 2023Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendationcited by 114
  4. 2018The Geometry of Culture: Analyzing Meaning through Word Embeddingscited by 87
  5. 2025Randomness, Not Representation: The Unreliability of Evaluating Cultural Alignment in LLMscited by 54
  6. 2024What's in a Name? Auditing Large Language Models for Race and Gender Biascited by 51
  7. 2025Normative Evaluation of Large Language Models with Everyday Moral Dilemmascited by 24
  8. 2025How Deep Is Representational Bias in LLMs? The Cases of Caste and Religioncited by 14
  9. 2022Trustworthy Social Bias Measurementcited by 7
  10. 2025Revealing Potential Biases in LLM-Based Recommender Systems in the Cold Start Settingcited by 6
  11. 2024Understanding the Interplay of Scale, Data, and Bias in Language Models: A Case Study with BERTcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Evaluating Multimodal Narrative Understanding of Popular Hollywood Films
  2. 2026Sympathetic Framing: Evaluating AI Alignment across Sociodemographic Groups
  3. 2025Exposing Blindspots: Cultural Bias Evaluation in Generative Image Models
  4. 2026Narrative Sharpens Gender Gaps: Surveying Film Characters with LLM Agents
  5. 2025The simulation of judgment in LLMs
  6. 2026Beyond Marginal Distributions: A Framework to Evaluate the Representativeness of Demographic-Aligned LLMs
  7. 2026Creative Collision: Directorial Persona Steering and Competition in Large Language Models
  8. 2025Multilingual != Multicultural: Evaluating Gaps Between Multilingual Capabilities and Cultural Alignment in LLMs