# FedEAT: A Robustness Optimization Framework for Federated LLMs

arXiv: https://arxiv.org/abs/2502.11863 · HTML: https://mlsanity.com/paper/2502.11863/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 24 references that are in the map.
- First posted on arXiv 17 Feb 2025.

## Builds on (most cited first, 20 of 20)

- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [Large Language Models: A Survey](https://mlsanity.com/paper/2402.06196/) (2024)
- [Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment](https://mlsanity.com/paper/2308.05374/) (2023)
- [A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly](https://mlsanity.com/paper/2312.02003/) (2023)
- [Analyzing Federated Learning through an Adversarial Lens](https://mlsanity.com/paper/1811.12470/) (2018)
- [Robust Aggregation for Federated Learning](https://mlsanity.com/paper/1912.13445/) (2019)
- [TrustLLM: Trustworthiness in Large Language Models](https://mlsanity.com/paper/2401.05561/) (2024)
- [Recent Advances in Adversarial Training for Adversarial Robustness](https://mlsanity.com/paper/2102.01356/) (2021)
- [Using an LLM to Help With Code Understanding](https://mlsanity.com/paper/2307.08177/) (2023)
- [Privacy and Robustness in Federated Learning: Attacks and Defenses](https://mlsanity.com/paper/2012.06337/) (2020)
- [The Science of Detecting LLM-Generated Texts](https://mlsanity.com/paper/2303.07205/) (2023)
- [Efficient Adversarial Training in LLMs with Continuous Attacks](https://mlsanity.com/paper/2405.15589/) (2024)
- [Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmark](https://mlsanity.com/paper/2311.06750/) (2023)
- [Robust Federated Learning in a Heterogeneous Environment](https://mlsanity.com/paper/1906.06629/) (2019)
- [Learning to Detect Malicious Clients for Robust Federated Learning](https://mlsanity.com/paper/2002.00211/) (2020)
- [Integration of Large Language Models and Federated Learning](https://mlsanity.com/paper/2307.08925/) (2023)
- [The Future of Large Language Model Pre-training is Federated](https://mlsanity.com/paper/2405.10853/) (2024)
- [Delving into the Adversarial Robustness of Federated Learning](https://mlsanity.com/paper/2302.09479/) (2023)
- [FedMKT: Federated Mutual Knowledge Transfer for Large and Small Language Models](https://mlsanity.com/paper/2406.02224/) (2024)
- [Federated Co-tuning Framework for Large and Small Language Models](https://mlsanity.com/paper/2411.11707/) (2024)

## Built on by (most cited first, 3 of 3)

- [A Survey on Foundation Models for Personalized Federated Intelligence](https://mlsanity.com/paper/2505.06907/) (2025)
- [FedReFT: Federated Representation Fine-Tuning with All-But-Me Aggregation](https://mlsanity.com/paper/2508.20295/) (2025)
- [Beyond Confidence: The Rhythms of Reasoning in Generative Models](https://mlsanity.com/paper/2602.10816/) (2026)

## Abstract (arXiv)

Significant advancements have been made by Large Language Models (LLMs) in the domains of natural language understanding and automated content creation. However, they still face persistent problems, including substantial computational costs and inadequate availability of training data. The combination of Federated Learning (FL) and LLMs (federated LLMs) offers a solution by leveraging distributed data while protecting privacy, which positions it as an ideal choice for sensitive domains. However, Federated LLMs still suffer from robustness challenges, including data heterogeneity, malicious clients, and adversarial attacks, which greatly hinder their applications. We first introduce the robustness problems in federated LLMs, to address these challenges, we propose FedEAT (Federated Embedding space Adversarial Training), a novel framework that applies adversarial training in the embedding space of client LLM and employs a robust aggregation approach, specifically geometric median aggregation, to enhance the robustness of Federated LLMs. Our experiments demonstrate that FedEAT effectively improves the robustness of Federated LLMs with minimal performance loss.