# Analyzing Federated Learning through an Adversarial Lens

arXiv: https://arxiv.org/abs/1811.12470 · HTML: https://mlsanity.com/paper/1811.12470/

## Key facts

- Cited by 351 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Nov 2018.

## Built on by (most cited first, 30 of 351)

- [A Field Guide to Federated Optimization](https://mlsanity.com/paper/2107.06917/) (2021)
- [Advances in adversarial attacks and defenses in computer vision: A survey](https://mlsanity.com/paper/2108.00401/) (2021)
- [FLDetector: Defending Federated Learning Against Model Poisoning Attacks via Detecting Malicious Clients](https://mlsanity.com/paper/2207.09209/) (2022)
- [Back to the Drawing Board: A Critical Evaluation of Poisoning Attacks on Production Federated Learning](https://mlsanity.com/paper/2108.10241/) (2021)
- [Truth Serum: Poisoning Machine Learning Models to Reveal Their Secrets](https://mlsanity.com/paper/2204.00032/) (2022)
- [Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmark](https://mlsanity.com/paper/2311.06750/) (2023)
- [Neurotoxin: Durable Backdoors in Federated Learning](https://mlsanity.com/paper/2206.10341/) (2022)
- [Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning](https://mlsanity.com/paper/2205.01992/) (2022)
- [CRFL: Certifiably Robust Federated Learning against Backdoor Attacks](https://mlsanity.com/paper/2106.08283/) (2021)
- [Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges](https://mlsanity.com/paper/2201.08135/) (2022)
- [CAFE: Catastrophic Data Leakage in Vertical Federated Learning](https://mlsanity.com/paper/2110.15122/) (2021)
- [Federated Unlearning with Knowledge Distillation](https://mlsanity.com/paper/2201.09441/) (2022)
- [FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learning](https://mlsanity.com/paper/2102.02514/) (2021)
- [BACKDOORL: Backdoor Attack against Competitive Reinforcement Learning](https://mlsanity.com/paper/2105.00579/) (2021)
- [MPAF: Model Poisoning Attacks to Federated Learning based on Fake Clients](https://mlsanity.com/paper/2203.08669/) (2022)
- [From Distributed Machine Learning to Federated Learning: A Survey](https://mlsanity.com/paper/2104.14362/) (2021)
- [Provably Secure Federated Learning against Malicious Clients](https://mlsanity.com/paper/2102.01854/) (2021)
- [Property Inference From Poisoning](https://mlsanity.com/paper/2101.11073/) (2021)
- [FedRecover: Recovering from Poisoning Attacks in Federated Learning using Historical Information](https://mlsanity.com/paper/2210.10936/) (2022)
- [Decentralized Federated Learning through Proxy Model Sharing](https://mlsanity.com/paper/2111.11343/) (2021)
- [Adversarial Attacks On Multi-Agent Communication](https://mlsanity.com/paper/2101.06560/) (2021)
- [SparseFed: Mitigating Model Poisoning Attacks in Federated Learning with Sparsification](https://mlsanity.com/paper/2112.06274/) (2021)
- [Survey: Leakage and Privacy at Inference Time](https://mlsanity.com/paper/2107.01614/) (2021)
- [Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts](https://mlsanity.com/paper/2002.04013/) (2020)
- [An Overview of Backdoor Attacks Against Deep Neural Networks and Possible Defences](https://mlsanity.com/paper/2111.08429/) (2021)
- [Recent Advances on Federated Learning: A Systematic Survey](https://mlsanity.com/paper/2301.01299/) (2023)
- [PipAttack: Poisoning Federated Recommender Systems forManipulating Item Promotion](https://mlsanity.com/paper/2110.10926/) (2021)
- [FLIP: A Provable Defense Framework for Backdoor Mitigation in Federated Learning](https://mlsanity.com/paper/2210.12873/) (2022)
- [Federated Learning for Malware Detection in IoT Devices](https://mlsanity.com/paper/2104.09994/) (2021)
- [FedIPR: Ownership Verification for Federated Deep Neural Network Models](https://mlsanity.com/paper/2109.13236/) (2021)

## Abstract (arXiv)

Federated learning distributes model training among a multitude of agents, who, guided by privacy concerns, perform training using their local data but share only model parameter updates, for iterative aggregation at the server. In this work, we explore the threat of model poisoning attacks on federated learning initiated by a single, non-colluding malicious agent where the adversarial objective is to cause the model to misclassify a set of chosen inputs with high confidence. We explore a number of strategies to carry out this attack, starting with simple boosting of the malicious agent's update to overcome the effects of other agents' updates. To increase attack stealth, we propose an alternating minimization strategy, which alternately optimizes for the training loss and the adversarial objective. We follow up by using parameter estimation for the benign agents' updates to improve on attack success. Finally, we use a suite of interpretability techniques to generate visual explanations of model decisions for both benign and malicious models and show that the explanations are nearly visually indistinguishable. Our results indicate that even a highly constrained adversary can carry out model poisoning attacks while simultaneously maintaining stealth, thus highlighting the vulnerability of the federated learning setting and the need to develop effective defense strategies.