mlsanityData to 14 Oct 2026

A Survey on Foundation Models for Personalized Federated Intelligence

arXiv 2505.06907 ↗First posted 11 May 2025cs.AI cs.CV cs.NEcited by 3

Not on the map. Why?

Abstract (arXiv)

The rise of large language models (LLMs), such as ChatGPT, Gemini, and Grok, has reshaped the AI landscape. As prominent instances of foundational models (FMs), they exhibit remarkable capabilities in generating human-like content, pushing the boundaries towards artificial general intelligence (AGI). However, their large-scale nature, privacy sensitivity, and substantial computational demands pose significant challenges for personalized customization for end users. To bridge this gap, we present the vision of artificial personalized intelligence (API), which focuses on adapting FMs to individual users while ensuring privacy. As a central enabler of API, we propose personalized federated intelligence (PFI), a new paradigm that not only integrates the privacy benefits of federated learning (FL) with the generalization capabilities of FMs but also places personalization at its core. To this end, we first survey recent advances in FL and FMs that lay the foundation for PFI. We then explore core stages of the PFI pipeline: efficient personalization at the edge, trustworthy adaptation, and adaptive refinement via retrieval-augmented generation. Finally, we highlight future directions for enabling PFI. Overall, this survey aims to lay a foundation for the development of API as a complementary direction to AGI, with PFI as a key enabling paradigm.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 106 of its 176 references that are in the map. First posted on arXiv 11 May 2025.

Builds on

106 of 176 references.

References · 106

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2022Training language models to follow instructions with human feedbackcited by 15,971
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  6. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  8. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  9. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  10. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  11. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  12. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
38 more
  1. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  2. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  4. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  5. 2019Advances and Open Problems in Federated Learningcited by 3,046
  6. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  7. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  8. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  9. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  10. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  11. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  12. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  13. 2019Deep Leakage from Gradientscited by 1,021
  14. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
  15. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
  16. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
  17. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 867
  18. 2021Scaling Vision with Sparse Mixture of Expertscited by 747
  19. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 703
  20. 2020Variational Quantum Algorithmscited by 692
  21. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  22. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 656
  23. 2024TinyLlama: An Open-Source Small Language Modelcited by 618
  24. 2019Federated Learning with Personalization Layerscited by 541
  25. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 536
  26. 2020An Efficient Framework for Clustered Federated Learningcited by 511
  27. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 463
  28. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
  29. 2019Fair Resource Allocation in Federated Learningcited by 394
  30. 2023A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPTcited by 368
  31. 2021Membership Inference Attacks on Machine Learning: A Surveycited by 321
  32. 2021A Field Guide to Federated Optimizationcited by 315
  33. 2022PromptSource: An Integrated Development Environment and Repository for Natural Language Promptscited by 289
  34. 2023A Survey on Model Compression for Large Language Modelscited by 269
  35. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 228
  36. 2023Segment Anything in Medical Imagescited by 227
  37. 2018An Introduction to Deep Reinforcement Learningcited by 206
  38. 2024Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decodingcited by 203

50 most cited of 106 All 106 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Multi-Modal Multi-Task Federated Foundation Models for Next-Generation Extended Reality Systems: Towards Privacy-Preserving Distributed Intelligence in AR/VR/MRcited by 3
  2. 2026Silent Failures in Federated Personalization of Foundation Modelscited by 0
  3. 2026Byzantine-Resilient Federated Large Language Model via Hypergraph Message Passingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020PFL-MoE: Personalized Federated Learning Based on Mixture of Experts
  2. 2025PM-MOE: Mixture of Experts on Private Model Parameters for Personalized Federated Learning
  3. 2024FedCoT: Federated Chain-of-Thought Distillation for Large Language Models
  4. 2024FedMoE: Personalized Federated Learning via Heterogeneous Mixture of Experts
  5. 2023PerAda: Parameter-Efficient Federated Learning Personalization with Generalization Guarantees
  6. 2024Recent Advances in Federated Learning Driven Large Language Models: A Survey on Architecture, Performance, and Security
  7. 2026FedCoE: Bridging Generalization and Personalization via Federated Coordinated Dual-level MoEs
  8. 2025Co-LoRA: Collaborative Model Personalization on Heterogeneous Multi-Modal Clients