# Fed-ADE: Adaptive Learning Rate for Federated Post-adaptation under Distribution Shift

arXiv: https://arxiv.org/abs/2603.01040 · HTML: https://mlsanity.com/paper/2603.01040/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 69 references that are in the map.
- First posted on arXiv 1 Mar 2026.

## Builds on (most cited first, 30 of 42)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016)
- [Model-Contrastive Federated Learning](https://mlsanity.com/paper/2103.16257/) (2021)
- [Ditto: Fair and Robust Federated Learning Through Personalization](https://mlsanity.com/paper/2012.04221/) (2020)
- [Exploiting Shared Representations for Personalized Federated Learning](https://mlsanity.com/paper/2102.07078/) (2021)
- [ReAct: Out-of-distribution Detection With Rectified Activations](https://mlsanity.com/paper/2111.12797/) (2021)
- [Detecting and Correcting for Label Shift with Black Box Predictors](https://mlsanity.com/paper/1802.03916/) (2018)
- [Think Locally, Act Globally: Federated Learning with Local and Global Representations](https://mlsanity.com/paper/2001.01523/) (2020)
- [Adaptive Gradient Methods with Dynamic Bound of Learning Rate](https://mlsanity.com/paper/1902.09843/) (2019)
- [Non-stationary Stochastic Optimization](https://mlsanity.com/paper/1307.5449/) (2013)
- [Federated Learning with Partial Model Personalization](https://mlsanity.com/paper/2204.03809/) (2022)
- [Meta-Learning with Adaptive Hyperparameters](https://mlsanity.com/paper/2011.00209/) (2020)
- [Examining and Combating Spurious Features under Distribution Shift](https://mlsanity.com/paper/2106.07171/) (2021)
- [FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learning](https://mlsanity.com/paper/2308.12305/) (2023)
- [Online Adaptation to Label Distribution Shift](https://mlsanity.com/paper/2107.04520/) (2021)
- [Momentum-Net: Fast and convergent iterative neural network for inverse problems](https://mlsanity.com/paper/1907.11818/) (2019)
- [R-TPT: Improving Adversarial Robustness of Vision-Language Models through Test-Time Prompt Tuning](https://mlsanity.com/paper/2504.11195/) (2025)
- [Adapting to Online Label Shift with Provable Guarantees](https://mlsanity.com/paper/2207.02121/) (2022)
- [Taxonomy of Machine Learning Safety: A Survey and Primer](https://mlsanity.com/paper/2106.04823/) (2021)
- [Adaptive Test-Time Personalization for Federated Learning](https://mlsanity.com/paper/2310.18816/) (2023)
- [Label Shift Adapter for Test-Time Adaptation under Covariate and Label Shifts](https://mlsanity.com/paper/2308.08810/) (2023)
- [Estimating and Explaining Model Performance When Both Covariates and Labels Shift](https://mlsanity.com/paper/2209.08436/) (2022)
- [Data Valuation and Detections in Federated Learning](https://mlsanity.com/paper/2311.05304/) (2023)
- [FedHCA2: Towards Hetero-Client Federated Multi-Task Learning](https://mlsanity.com/paper/2311.13250/) (2023)
- [Adapting to Continuous Covariate Shift via Online Density Ratio Estimation](https://mlsanity.com/paper/2302.02552/) (2023)
- [Performative Federated Learning: A Solution to Model-Dependent and Heterogeneous Distribution Shifts](https://mlsanity.com/paper/2305.05090/) (2023)

## Built on by (most cited first, 2 of 2)

- [Multi2: Hierarchical Multi-Agent Decision-Making with LLM-Based Agents in Interactive Environments](https://mlsanity.com/paper/2606.03698/) (2026)
- [Post-Hoc Merging is Not Enough: Many-Shot Model Merging with Loss-Gap Balancing](https://mlsanity.com/paper/2606.16501/) (2026)

## Abstract (arXiv)

Federated learning (FL) in post-deployment settings must adapt to non-stationary data streams across heterogeneous clients without access to ground-truth labels. A major challenge is learning rate selection under client-specific, time-varying distribution shifts, where fixed learning rates often lead to underfitting or divergence. We propose Fed-ADE (Federated Adaptation with Distribution Shift Estimation), an unsupervised federated adaptation framework that leverages lightweight estimators of distribution dynamics. Specifically, Fed-ADE employs uncertainty dynamics estimation to capture changes in predictive uncertainty and representation dynamics estimation to detect covariate-level feature drift, combining them into a per-client, per-timestep adaptive learning rate. We provide theoretical analyses showing that our dynamics estimation approximates the underlying distribution shift and yields dynamic regret and convergence guarantees. Experiments on image and text benchmarks under diverse distribution shifts (label and covariate) demonstrate consistent improvements over strong baselines. These results highlight that distribution shift-aware adaptation enables effective and robust federated post-adaptation under real-world non-stationarity.