mlsanityData to 7 Oct 2026

Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Security

arXiv 2401.05459 ↗First posted 10 Jan 2024cs.HC cs.AI cs.SEcited by 257

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Since the advent of personal computing devices, intelligent personal assistants (IPAs) have been one of the key technologies that researchers and engineers have focused on, aiming to help users efficiently obtain information and execute tasks, and provide users with more intelligent, convenient, and rich interaction experiences. With the development of smartphones and IoT, computing and sensing devices have become ubiquitous, greatly expanding the boundaries of IPAs. However, due to the lack of capabilities such as user intent understanding, task planning, tool using, and personal data management etc., existing IPAs still have limited practicality and scalability. Recently, the emergence of foundation models, represented by large language models (LLMs), brings new opportunities for the development of IPAs. With the powerful semantic understanding and reasoning capabilities, LLM can enable intelligent agents to solve complex problems autonomously. In this paper, we focus on Personal LLM Agents, which are LLM-based agents that are deeply integrated with personal data and personal devices and used for personal assistance. We envision that Personal LLM Agents will become a major software paradigm for end-users in the upcoming era. To realize this vision, we take the first step to discuss several important questions about Personal LLM Agents, including their architecture, capability, efficiency and security. We start by summarizing the key components and design choices in the architecture of Personal LLM Agents, followed by an in-depth analysis of the opinions collected from domain experts. Next, we discuss several key challenges to achieve intelligent, efficient and secure Personal LLM Agents, followed by a comprehensive survey of representative solutions to address these challenges.

Key facts. Cited by 257 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 187 of its 385 references that are in the map. First posted on arXiv 10 Jan 2024.

Builds on

187 of 385 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023OpenAgents: An Open Platform for Language Agents in the Wild16 Oct
  2. 2023FireAct: Toward Language Agent Fine-tuning9 Oct
  3. 2023Agents: An Open-source Framework for Autonomous Language Agents14 Sep
  4. 2023The Rise and Potential of Large Language Model Based Agents: A Survey14 Sep
  5. 2023Cognitive Architectures for Language Agents5 Sep

Other references · 182

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  8. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  9. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  10. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  11. 2020Scaling Laws for Neural Language Modelscited by 6,411
  12. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
38 more
  1. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  2. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  5. 2017Deep reinforcement learning from human preferencescited by 3,846
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  7. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  8. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  9. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  10. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  11. 2023A Survey of Large Language Modelscited by 2,537
  12. 2020Extracting Training Data from Large Language Modelscited by 2,144
  13. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  14. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  15. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  16. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  17. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  18. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  19. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  20. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  21. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  22. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  23. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  24. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  25. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  26. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  27. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  28. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  29. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  30. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  31. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  32. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  33. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  34. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  35. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  36. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
  37. 2023Teaching Large Language Models to Self-Debugcited by 963
  38. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895

50 most cited of 182 All 182 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 77; 2025: 117; 2026: 63.

257 papers.

Sort:

By area:

  1. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  2. 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agentscited by 268
  3. 2024A Survey on Efficient Inference for Large Language Modelscited by 197
  4. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 194
  5. 2024Large Language Model-Brained GUI Agents: A Surveycited by 160
  6. 2025From LLM Reasoning to Autonomous AI Agents: A Comprehensive Reviewcited by 146
  7. 2025Evaluation and Benchmarking of LLM Agents: A Surveycited by 145
  8. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 131
  9. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Modelscited by 129
  10. 2024Mobile Edge Intelligence for Large Language Models: A Contemporary Surveycited by 125
  11. 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 122
  12. 2025Progent: Securing AI Agents with Privilege Controlcited by 121
38 more
  1. 2024LLM Multi-Agent Systems: Challenges and Open Problemscited by 119
  2. 2025In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agentscited by 102
  3. 2024On-Device Language Models: A Comprehensive Reviewcited by 101
  4. 2025A Survey on Trustworthy LLM Agents: Threats and Countermeasurescited by 93
  5. 2025KVzip: Query-Agnostic KV Cache Compression with Context Reconstructioncited by 89
  6. 2025A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Modelscited by 87
  7. 2024A Survey of Resource-efficient LLM and Multimodal Foundation Modelscited by 84
  8. 2024Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistancecited by 84
  9. 2025MemBench: Towards More Comprehensive Evaluation on the Memory of LLM-based Agentscited by 82
  10. 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 78
  11. 2024From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agentscited by 76
  12. 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoningcited by 72
  13. 2024Retrieval-Augmented Generation for Natural Language Processing: A Surveycited by 70
  14. 2024TableGPT2: A Large Multimodal Model with Tabular Data Integrationcited by 69
  15. 2025Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agentscited by 66
  16. 2025A Survey of Personalized Large Language Models: Progress and Future Directionscited by 65
  17. 2024MobileAgentBench: An Efficient and User-Friendly Benchmark for Mobile LLM Agentscited by 63
  18. 2024KnowAgent: Knowledge-Augmented Planning for LLM-Based Agentscited by 61
  19. 2025Hierarchical Memory for High-Efficiency Long-Term Reasoning in LLM Agentscited by 60
  20. 2024HYDRA: Model Factorization Framework for Black-Box LLM Personalizationcited by 59
  21. 2024Fast On-device LLM Inference with NPUscited by 58
  22. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 54
  23. 2024HARGPT: Are LLMs Zero-Shot Human Activity Recognizers?cited by 52
  24. 2026Taming Various Privilege Escalation in LLM-Based Agent Systems: A Mandatory Access Control Frameworkcited by 50
  25. 2025Measuring What Makes You Unique: Difference-Aware User Modeling for Enhancing LLM Personalizationcited by 44
  26. 2025Agentic Web: Weaving the Next Web with AI Agentscited by 43
  27. 2025A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applicationscited by 40
  28. 2025Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AIcited by 39
  29. 2025AgentArmor: Enforcing Program Analysis on Agent Runtime Trace to Defend Against Prompt Injectioncited by 39
  30. 2024Crafting Personalized Agents through Retrieval-Augmented Generation on Editable Memory Graphscited by 38
  31. 2025LiCoMemory: Lightweight and Cognitive Agentic Memory for Efficient Long-Term Reasoningcited by 37
  32. 2024LlamaTouch: A Faithful and Scalable Testbed for Mobile UI Task Automationcited by 36
  33. 2024Richelieu: Self-Evolving LLM-Based Agents for AI Diplomacycited by 36
  34. 2024Benchmarking Agentic Workflow Generationcited by 36
  35. 2024HackSynth: LLM Agent and Evaluation Framework for Autonomous Penetration Testingcited by 35
  36. 2025Position: Episodic Memory is the Missing Piece for Long-Term LLM Agentscited by 35
  37. 2025OS Agents: A Survey on MLLM-based Agents for General Computing Devices Usecited by 35
  38. 2024AutoDroid-V2: Boosting SLM-based GUI Agents via Code Generationcited by 31

50 most cited of 257 All 257 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directions
  2. 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studies
  3. 2024Creating an LLM-based AI-agent: A high-level methodology towards enhancing LLMs with APIs
  4. 2025From Assistants to Adversaries: Exploring the Security Risks of Mobile LLM Agents
  5. 2025LLM-Based Human-Agent Collaboration and Interaction Systems: A Survey
  6. 2026Multi-User Large Language Model Agents
  7. 2023A Survey on Large Language Model based Autonomous Agents
  8. 2024Understanding the Weakness of Large Language Model Agents within a Complex Android Environment