# Federated In-Context LLM Agent Learning

arXiv: https://arxiv.org/abs/2412.08054 · HTML: https://mlsanity.com/paper/2412.08054/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 10 of its 28 references that are in the map.
- First posted on arXiv 11 Dec 2024.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [Agentic AI for wireless networks](https://mlsanity.com/agents/agentic-ai-for-wireless-networks/)

## Builds on (most cited first, 10 of 10)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model](https://mlsanity.com/paper/2405.04434/) (2024)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [C-Pack: Packed Resources For General Chinese Embeddings](https://mlsanity.com/paper/2309.07597/) (2023)
- [ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases](https://mlsanity.com/paper/2306.05301/) (2023)
- [Improving LoRA in Privacy-preserving Federated Learning](https://mlsanity.com/paper/2403.12313/) (2024)
- [Tree of Clarifications: Answering Ambiguous Questions with Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2310.14696/) (2023)
- [Counting-Stars: A Multi-evidence, Position-aware, and Scalable Benchmark for Evaluating Long-Context Large Language Models](https://mlsanity.com/paper/2403.11802/) (2024)
- [FedPFT: Federated Proxy Fine-Tuning of Foundation Models](https://mlsanity.com/paper/2404.11536/) (2024)
- [RePrompt: Planning by Automatic Prompt Engineering for Large Language Models Agents](https://mlsanity.com/paper/2406.11132/) (2024)

## Built on by (most cited first, 10 of 10)

- [Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems](https://mlsanity.com/paper/2504.01990/) (2025)
- [Federated In-Context Learning: Iterative Refinement for Improved Answer Quality](https://mlsanity.com/paper/2506.07440/) (2025)
- [A Survey on Foundation Models for Personalized Federated Intelligence](https://mlsanity.com/paper/2505.06907/) (2025)
- [Fed-SE: Federated Self-Evolution for Privacy-Constrained Multi-Environment LLM Agents](https://mlsanity.com/paper/2512.08870/) (2025)
- [Typed Federated Artifacts for the Agentic Web:Sharing Tool-Routing Knowledge Across Frozen,Heterogeneous LLM Agents](https://mlsanity.com/paper/2609.06815/) (2026)
- [FERA: Uncertainty-Aware Federated Reasoning for Large Language Models](https://mlsanity.com/paper/2605.10082/) (2026)
- [AsynDBT: Asynchronous Distributed Bilevel Tuning for efficient In-Context Learning with Large Language Models](https://mlsanity.com/paper/2602.17694/) (2026)
- [Federated Agent Optimization](https://mlsanity.com/paper/2610.01195/) (2026)
- [PFedMA: Who Should Join Next as Federated Multi-Agent Orchestration Evolves?](https://mlsanity.com/paper/2610.19458/) (2026)
- [XferAgent: Cross-Agent Experience Transfer for Self-Improving LLM Agents](https://mlsanity.com/paper/2610.19581/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have revolutionized intelligent services by enabling logical reasoning, tool use, and interaction with external systems as agents. The advancement of LLMs is frequently hindered by the scarcity of high-quality data, much of which is inherently sensitive. Federated learning (FL) offers a potential solution by facilitating the collaborative training of distributed LLMs while safeguarding private data. However, FL frameworks face significant bandwidth and computational demands, along with challenges from heterogeneous data distributions. The emerging in-context learning capability of LLMs offers a promising approach by aggregating natural language rather than bulky model parameters. Yet, this method risks privacy leakage, as it necessitates the collection and presentation of data samples from various clients during aggregation. In this paper, we propose a novel privacy-preserving Federated In-Context LLM Agent Learning (FICAL) algorithm, which to our best knowledge for the first work unleashes the power of in-context learning to train diverse LLM agents through FL. In our design, knowledge compendiums generated by a novel LLM-enhanced Knowledge Compendiums Generation (KCG) module are transmitted between clients and the server instead of model parameters in previous FL methods. Apart from that, an incredible Retrieval Augmented Generation (RAG) based Tool Learning and Utilizing (TLU) module is designed and we incorporate the aggregated global knowledge compendium as a teacher to teach LLM agents the usage of tools. We conducted extensive experiments and the results show that FICAL has competitive performance compared to other SOTA baselines with a significant communication cost decrease of 3.33×105 times.