# StreamProfileBench: A Benchmark for Fine-Grained User Profile Inference in Real-World Streaming Scenarios

arXiv: https://arxiv.org/abs/2605.25758 · HTML: https://mlsanity.com/paper/2605.25758/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 40 references that are in the map.
- First posted on arXiv 25 May 2026.

## Builds on (most cited first, 27 of 27)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [LaMP: When Large Language Models Meet Personalization](https://mlsanity.com/paper/2304.11406/) (2023)
- [Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)](https://mlsanity.com/paper/2203.13366/) (2022)
- [TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation](https://mlsanity.com/paper/2305.00447/) (2023)
- [Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System](https://mlsanity.com/paper/2303.14524/) (2023)
- [Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scale](https://mlsanity.com/paper/2504.14225/) (2025)
- [Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs](https://mlsanity.com/paper/2502.09597/) (2025)
- [PersonaMem-v2: Towards Personalized Intelligence via Learning Implicit User Personas and Agentic Memory](https://mlsanity.com/paper/2512.06688/) (2025)
- [Measuring What Makes You Unique: Difference-Aware User Modeling for Enhancing LLM Personalization](https://mlsanity.com/paper/2503.02450/) (2025)
- [SentimentGPT: Exploiting GPT for Advanced Sentiment Analysis and its Departure from Current Machine Learning](https://mlsanity.com/paper/2307.10234/) (2023)
- [PersonaX: A Recommendation Agent Oriented User Modeling Framework for Long Behavior Sequence](https://mlsanity.com/paper/2503.02398/) (2025)
- [HieRec: Hierarchical User Interest Modeling for Personalized News Recommendation](https://mlsanity.com/paper/2106.04408/) (2021)
- [NextQuill: Causal Preference Modeling for Enhancing LLM Personalization](https://mlsanity.com/paper/2506.02368/) (2025)
- [User Modeling and User Profiling: A Comprehensive Survey](https://mlsanity.com/paper/2402.09660/) (2024)
- [PersonaBench: Evaluating AI Models on Understanding Personal Information through Accessing (Synthetic) Private User Data](https://mlsanity.com/paper/2502.20616/) (2025)
- [Navigating Prompt Complexity for Zero-Shot Classification: A Study of Large Language Models in Computational Social Science](https://mlsanity.com/paper/2305.14310/) (2023)
- [RecoWorld: Building Simulated Environments for Agentic Recommender Systems](https://mlsanity.com/paper/2509.10397/) (2025)
- [LettinGo: Explore User Profile Generation for Recommendation System](https://mlsanity.com/paper/2506.18309/) (2025)
- [Towards Explainable Temporal User Profiling with LLMs](https://mlsanity.com/paper/2505.00886/) (2025)
- [Prompt Tuning as User Inherent Profile Inference Machine](https://mlsanity.com/paper/2408.06577/) (2024)
- [User Profile with Large Language Models: Construction, Updating, and Benchmarking](https://mlsanity.com/paper/2502.10660/) (2025)
- [Ladder-of-Thought: Using Knowledge as Steps to Elevate Stance Detection](https://mlsanity.com/paper/2308.16763/) (2023)
- [EdgeWisePersona: A Dataset for On-Device User Profiling from Natural Language Interactions](https://mlsanity.com/paper/2505.11417/) (2025)
- [Conf-Profile: A Confidence-Driven Reasoning Paradigm for Label-Free User Profiling](https://mlsanity.com/paper/2509.18864/) (2025)
- [ALPBench: A Benchmark for Attribution-level Long-term Personal Behavior Understanding](https://mlsanity.com/paper/2602.03056/) (2026)
- [ITDR: An Instruction Tuning Dataset for Enhancing Large Language Models in Recommendations](https://mlsanity.com/paper/2508.05667/) (2025)

## Built on by (most cited first, 1 of 1)

- [Vela: User Behavior as Supervision Sparks LLM Intelligence](https://mlsanity.com/paper/2610.14709/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have reshaped user profiling, yet current evaluations mainly focus on static data snapshots. This paradigm overlooks the reality of personalized systems, where User-Generated Content (UGC) arrives continuously and fine-grained profiles evolve rapidly. To bridge this gap, we introduce StreamProfileBench, a large-scale benchmark for fine-grained streaming user profiling. We formalize streaming user profiling as a continuous state maintenance task and curate a highly authentic dataset comprising over 120,000 UGC posts from 7,000+ real users across five diverse platforms. By leveraging the temporal correlation of user interests, we further propose a novel, annotation-free evaluation framework. Extensive experiments across 14 leading LLMs reveal that continuous profile updating remains an open challenge. Models exhibit a systemic conservative bias, over-retaining past interests while failing to recognize interest decay. Ablation experiments further validate the practical utility and necessity of the streaming paradigm. Data and code are hosted in https://github.com/WaterWang-001/StreamProfileBench.