mlsanityData to 7 Oct 2026

Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoders

arXiv 2403.03952 ↗First posted 6 Mar 2024cs.IRcited by 213

Not on the map. Why?

Cited by 21 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Feature engineering has long been central to recommender systems, yet effectively leveraging textual item features remains challenging. Recent advances in large language models (LLMs) have enabled their use as semantic encoders for recommendation, but their roles and behaviors in this setting are still not well understood. Prior studies often rely on general-purpose embedding benchmarks (e.g., MTEB) when selecting LLMs, overlooking the unique characteristics of recommendation tasks. To address this gap, we introduce BLaIR, a comprehensive benchmark for evaluating LLMs as semantic encoders in recommendation scenarios. We contribute (1) a new large-scale Amazon Reviews 2023 dataset with over 570 million reviews and 48 million items, (2) a unified benchmark covering sequential recommendation, collaborative filtering, and product search, and (3) a new complex-query product search task featuring both semi-synthetic and real-world evaluation datasets. Experiments with 11 leading LLMs show that their rankings on BLaIR show little correlation with MTEB, highlighting the unique challenges of semantic encoding in recommendation.

Key facts. Cited by 213 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Mar 2024.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 38; 2025: 117; 2026: 58.

213 papers.

Sort:

By area:

  1. 2025RM-R1: Reward Modeling as Reasoningcited by 149
  2. 2024Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferencescited by 142
  3. 2024Large Language Models Empowered Personalized Web Agentscited by 64
  4. 2024Caution for the Environment: Multimodal LLM Agents are Susceptible to Environmental Distractionscited by 55
  5. 2025Measuring What Makes You Unique: Difference-Aware User Modeling for Enhancing LLM Personalizationcited by 44
  6. 2025Implicit Reasoning in Large Language Models: A Comprehensive Surveycited by 42
  7. 2025Rec-R1: Bridging Generative Large Language Models and User-Centric Recommendation Systems via Reinforcement Learningcited by 32
  8. 2024Mosaic of Modalities: A Comprehensive Benchmark for Multimodal Graph Learningcited by 28
  9. 2024Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Predictioncited by 28
  10. 2025OPeRA: A Dataset of Observation, Persona, Rationale, and Action for Evaluating LLMs on Human Online Shopping Behavior Simulationcited by 26
  11. 2025AgentCF++: Memory-enhanced LLM-based Agents for Popularity-aware Cross-domain Recommendationscited by 23
  12. 2025From Calibration to Collaboration: LLM Uncertainty Quantification Should Be More Human-Centeredcited by 21
38 more
  1. 2024Review-driven Personalized Preference Reasoning with Large Language Models for Recommendationcited by 20
  2. 2024Shopping MMLU: A Massive Multi-Task Online Shopping Benchmark for Large Language Modelscited by 20
  3. 2025Reason-to-Recommend: Using Interaction-of-Thought Reasoning to Enhance LLM Recommendationcited by 20
  4. 2025Latent Inter-User Difference Modeling for LLM Personalizationcited by 20
  5. 2025LLM2Rec: Large Language Models Are Powerful Embedding Models for Sequential Recommendationcited by 19
  6. 2024Language Representations Can be What Recommenders Need: Findings and Potentialscited by 18
  7. 2025ActionPiece: Contextually Tokenizing Action Sequences for Generative Recommendationcited by 18
  8. 2025MiniOneRec: An Open-Source Framework for Scaling Generative Recommendationcited by 16
  9. 2026MemoryCD: Benchmarking Long-Context User Memory of LLM Agents for Lifelong Cross-Domain Personalizationcited by 15
  10. 2025Can Language Models Follow Multiple Turns of Entangled Instructions?cited by 14
  11. 2024A Survey of Controllable Learning: Methods and Applications in Information Retrievalcited by 12
  12. 2024A Multimodal Single-Branch Embedding Network for Recommendation in Cold-Start and Missing Modality Scenarioscited by 12
  13. 2025Tokenization is Sensitive to Language Variationcited by 12
  14. 2025AgentSociety Challenge: Designing LLM Agents for User Modeling and Recommendation on Web Platformscited by 12
  15. 2026Reasoning over Semantic IDs Enhances Generative Recommendationcited by 12
  16. 2024Make Large Language Model a Better Rankercited by 11
  17. 2025Pluralistic Off-policy Evaluation and Alignmentcited by 11
  18. 2024EasyRec: Simple yet Effective Language Models for Recommendationcited by 10
  19. 2024End-to-end Training for Recommendation with Language-based User Profilescited by 10
  20. 2024Advancing Content Moderation: Evaluating Large Language Models for Detecting Sensitive Content Across Text, Images, and Videoscited by 10
  21. 2024SmartAgent: Chain-of-User-Thought for Embodied Personalized Agent in Cyber Worldcited by 10
  22. 2025Personalized Graph-Based Retrieval for Large Language Modelscited by 10
  23. 2025Bias Beware: The Impact of Cognitive Biases on LLM-Driven Product Recommendationscited by 10
  24. 2025EcomScriptBench: A Multi-task Benchmark for E-commerce Script Planning via Step-wise Intention-Driven Product Associationcited by 10
  25. 2024TEG-DB: A Comprehensive Dataset and Benchmark of Textual-Edge Graphscited by 9
  26. 2025Improving LLM-powered Recommendations with Personalized Informationcited by 9
  27. 2025MR.Rec: Synergizing Memory and Reasoning for Personalized Recommendation Assistant with LLMscited by 9
  28. 2025Massive Memorization with Hundreds of Trillions of Parameters for Sequential Transducer Generative Recommenderscited by 9
  29. 2025LLaDA-Rec: Discrete Diffusion for Parallel Semantic ID Generation in Generative Recommendationcited by 9
  30. 2024A Multi-Faceted Evaluation Framework for Assessing Synthetic Data Generated by Large Language Modelscited by 8
  31. 2024Multi-domain Knowledge Graph Collaborative Pre-training and Prompt Tuning for Diverse Downstream Taskscited by 8
  32. 2024LLaSA: Large Language and E-Commerce Shopping Assistantcited by 8
  33. 2024Preference Diffusion for Recommendationcited by 8
  34. 2024Beyond Retrieval: Generating Narratives in Conversational Recommender Systemscited by 8
  35. 2025Lost in Sequence: Do Large Language Models Understand Sequential Recommendation?cited by 8
  36. 2025The Hidden Threat in Plain Text: Attacking RAG Data Loaderscited by 8
  37. 2025AbsTopK: Rethinking Sparse Autoencoders For Bidirectional Featurescited by 8
  38. 2026OneReason Technical Reportcited by 8

50 most cited of 213 All 213 (JSON)