# ProfiLLM: An LLM-Based Framework for Implicit Profiling of Chatbot Users

arXiv: https://arxiv.org/abs/2506.13980 · HTML: https://mlsanity.com/paper/2506.13980/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 33 references that are in the map.
- First posted on arXiv 16 Jun 2025.

## Builds on (most cited first, 9 of 9)

- [Let the LLMs Talk: Simulating Human-to-Human Conversational QA via Zero-Shot LLM-to-LLM Interactions](https://mlsanity.com/paper/2312.02913/) (2023)
- [ONCE: Boosting Content-based Recommendation with Both Open- and Closed-source Large Language Models](https://mlsanity.com/paper/2305.06566/) (2023)
- [One Chatbot Per Person: Creating Personalized Chatbots based on Implicit User Profiles](https://mlsanity.com/paper/2108.09355/) (2021)
- [User Modeling in the Era of Large Language Models: Current Research and Future Directions](https://mlsanity.com/paper/2312.11518/) (2023)
- [LLM Roleplay: Simulating Human-Chatbot Interaction](https://mlsanity.com/paper/2407.03974/) (2024)
- [Learning Implicit User Profiles for Personalized Retrieval-Based Chatbot](https://mlsanity.com/paper/2108.07935/) (2021)
- [ChatGPT for Suicide Risk Assessment on Social Media: Quantitative Evaluation of Model Performance, Potentials and Limitations](https://mlsanity.com/paper/2306.09390/) (2023)
- [Apollonion: Profile-centric Dialog Agent](https://mlsanity.com/paper/2404.08692/) (2024)
- [DiagGPT: An LLM-based and Multi-agent Dialogue System with Automatic Topic Management for Flexible Task-Oriented Dialogue](https://mlsanity.com/paper/2308.08043/) (2023)

## Built on by (most cited first, 3 of 3)

- [Is Passive Expertise-Based Personalization Enough? A Case Study in AI-Assisted Test-Taking](https://mlsanity.com/paper/2511.23376/) (2025)
- [SecMate: Multi-Agent Adaptive Cybersecurity Troubleshooting with Tri-Context Personalization](https://mlsanity.com/paper/2604.26394/) (2026)
- [Same Voice, Different Lab: On the Homogenization of Frontier LLM Personalities](https://mlsanity.com/paper/2605.02897/) (2026)

## Abstract (arXiv)

Despite significant advancements in conversational AI, large language model (LLM)-powered chatbots often struggle with personalizing their responses according to individual user characteristics, such as technical expertise, learning style, and communication preferences. This lack of personalization is particularly problematic in specialized knowledge-intense domains like IT/cybersecurity (ITSec), where user knowledge levels vary widely. Existing approaches for chatbot personalization primarily rely on static user categories or explicit self-reported information, limiting their adaptability to an evolving perception of the user's proficiency, obtained in the course of ongoing interactions. In this paper, we propose ProfiLLM, a novel framework for implicit and dynamic user profiling through chatbot interactions. This framework consists of a taxonomy that can be adapted for use in diverse domains and an LLM-based method for user profiling in terms of the taxonomy. To demonstrate ProfiLLM's effectiveness, we apply it in the ITSec domain where troubleshooting interactions are used to infer chatbot users' technical proficiency. Specifically, we developed ProfiLLM[ITSec], an ITSec-adapted variant of ProfiLLM, and evaluated its performance on 1,760 human-like chatbot conversations from 263 synthetic users. Results show that ProfiLLM[ITSec] rapidly and accurately infers ITSec profiles, reducing the gap between actual and predicted scores by up to 55--65% after a single prompt, followed by minor fluctuations and further refinement. In addition to evaluating our new implicit and dynamic profiling framework, we also propose an LLM-based persona simulation methodology, a structured taxonomy for ITSec proficiency, our codebase, and a dataset of chatbot interactions to support future research.