mlsanityData to 7 Oct 2026

AgentSociety Challenge: Designing LLM Agents for User Modeling and Recommendation on Web Platforms

arXiv 2502.18754 ↗First posted 26 Feb 2025cs.IR cs.AIcited by 12

On the map: LLM agents and tool use → Agentic recommender systems

Abstract (arXiv)

The AgentSociety Challenge is the first competition in the Web Conference that aims to explore the potential of Large Language Model (LLM) agents in modeling user behavior and enhancing recommender systems on web platforms. The Challenge consists of two tracks: the User Modeling Track and the Recommendation Track. Participants are tasked to utilize a combined dataset from Yelp, Amazon, and Goodreads, along with an interactive environment simulator, to develop innovative LLM agents. The Challenge has attracted 295 teams across the globe and received over 1,400 submissions in total over the course of 37 official competition days. The participants have achieved 21.9% and 20.3% performance improvement for Track 1 and Track 2 in the Development Phase, and 9.1% and 15.9% in the Final Phase, representing a significant accomplishment. This paper discusses the detailed designs of the Challenge, analyzes the outcomes, and highlights the most successful LLM agent designs. To support further research and development, we have open-sourced the benchmark environment at https://tsinghua-fib-lab.github.io/AgentSocietyChallenge.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Agentic recommender systems, one of 16 directions in LLM agents and tool use. Builds on 19 of its 21 references that are in the map. First posted on arXiv 26 Feb 2025.

Builds on

19 of 21 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023On Generative Agents in Recommendation16 Oct
  2. 2023AgentCF: Collaborative Learning with Autonomous Language Agents for Recommender Systems13 Oct
  3. 2023RecMind: Large Language Model Powered Agent For Recommendation28 Aug

Other references · 16

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  4. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  5. 2017Neural Collaborative Filteringcited by 1,108
  6. 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 414
  7. 2023A Survey on Large Language Models for Recommendationcited by 378
  8. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 291
  9. 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 253
  10. 2024Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoderscited by 213
  11. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
  12. 2024AgentSquare: Automatic LLM Agent Search in Modular Design Spacecited by 141
4 more
  1. 2016Yelp Dataset Challenge: Review Rating Predictioncited by 84
  2. 2023LLaRA: Large Language-Recommendation Assistantcited by 56
  3. 2025Division-of-Thoughts: Harnessing Hybrid Language Model Synergy for Efficient On-Device Agentscited by 24
  4. 2024Large Language Model-driven Meta-structure Discovery in Heterogeneous Information Networkcited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 4.

12 papers.

  1. 2025A Survey of Large Language Model Empowered Agents for Recommendation and Search: Towards Next-Generation Information Retrievalcited by 31
  2. 2025AgentRecBench: Benchmarking LLM Agent-based Personalized Recommender Systemscited by 23
  3. 2025FingerTip 20K: A Benchmark for Proactive and Personalized Mobile LLM Agentscited by 19
  4. 2025AgentSwift: Efficient LLM Agent Design via Value-guided Hierarchical Searchcited by 14
  5. 2026BalDRO: A Distributionally Robust Optimization based Framework for Large Language Model Unlearningcited by 8
  6. 2025TAI3: Testing Agent Integrity in Interpreting User Intentcited by 7
  7. 2026ScienceDB AI: An LLM-Driven Agentic Recommender System for Large-Scale Scientific Data Sharing Servicescited by 5
  8. 2025CrimeMind: Simulating Urban Crime with Multi-Modal LLM Agentscited by 2
  9. 2025SOCIA-nabla: Textual Gradient Meets Multi-Agent Orchestration for Automated Simulator Generationcited by 1
  10. 2025SOCIA-Nabla: Textual Gradient Meets Multi-Agent Orchestration for Automated Simulator Generationcited by 0
  11. 2026SOCIA-EVO: Automated Simulator Construction via Dual-Anchored Bi-Level Optimizationcited by 0
  12. 2026APeB: Benchmarking Personalization Ability of Large Language Model Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025SoMe: A Realistic Benchmark for LLM-based Social Media Agents
  2. 2025A Survey on LLM-powered Agents for Recommender Systems
  3. 2026AgentRecommender: LLM Agents Enable Customizable Recommender Systems on the User Side
  4. 2026Do Recommendation Algorithms Work When Users Are LLM Agents? A Case Study on Moltbook
  5. 2025Multi-Agent Collaborative Filtering: Orchestrating Users and Items for Agentic Recommendations
  6. 2024AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents
  7. 2026Is Your LLM-as-a-Recommender Agent Trustable? LLMs' Recommendation is Easily Hacked by Biases (Preferences)
  8. 2025Towards Agentic Recommender Systems in the Era of Multimodal Large Language Models