# FedEMA: Federated Exponential Moving Averaging with Negative Entropy Regularizer in Autonomous Driving

arXiv: https://arxiv.org/abs/2505.00318 · HTML: https://mlsanity.com/paper/2505.00318/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 39 references that are in the map.
- First posted on arXiv 1 May 2025.

## Builds on (most cited first, 25 of 25)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Model-Contrastive Federated Learning](https://mlsanity.com/paper/2103.16257/) (2021)
- [Federated Learning Based on Dynamic Regularization](https://mlsanity.com/paper/2111.04263/) (2021)
- [Personalized Cross-Silo Federated Learning on Non-IID Data](https://mlsanity.com/paper/2007.03797/) (2020)
- [Rethinking Semantic Segmentation: A Prototype View](https://mlsanity.com/paper/2203.15102/) (2022)
- [Federated Visual Classification with Real-World Data Distribution](https://mlsanity.com/paper/2003.08082/) (2020)
- [Federated Class-Incremental Learning](https://mlsanity.com/paper/2203.11473/) (2022)
- [Multimodal End-to-End Autonomous Driving](https://mlsanity.com/paper/1906.03199/) (2019)
- [TopFormer: Token Pyramid Transformer for Mobile Semantic Segmentation](https://mlsanity.com/paper/2204.05525/) (2022)
- [Understanding Bird's-Eye View of Road Semantics using an Onboard Camera](https://mlsanity.com/paper/2012.03040/) (2020)
- [WEDGE: Web-Image Assisted Domain Generalization for Semantic Segmentation](https://mlsanity.com/paper/2109.14196/) (2021)
- [pFedLVM: A Large Vision Model (LVM)-Driven and Latent Feature-Based Personalized Federated Learning Framework in Autonomous Driving](https://mlsanity.com/paper/2405.04146/) (2024)
- [Superpixel Transformers for Efficient Semantic Segmentation](https://mlsanity.com/paper/2309.16889/) (2023)
- [IDA: Informed Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2303.02741/) (2023)
- [FedRC: A Rapid-Converged Hierarchical Federated Learning Framework in Street Scene Semantic Understanding](https://mlsanity.com/paper/2407.01103/) (2024)
- [Fast-Convergent and Communication-Alleviated Heterogeneous Hierarchical Federated Learning in Autonomous Driving](https://mlsanity.com/paper/2409.19560/) (2024)
- [Communication Resources Constrained Hierarchical Federated Learning for End-to-End Autonomous Driving](https://mlsanity.com/paper/2306.16169/) (2023)
- [Semi-Supervised Active Learning for Semantic Segmentation in Unknown Environments Using Informative Path Planning](https://mlsanity.com/paper/2312.04402/) (2023)
- [Enhancing Large Vision Model in Street Scene Semantic Understanding through Leveraging Posterior Optimization Trajectory](https://mlsanity.com/paper/2501.01710/) (2025)
- [FFINet: Future Feedback Interaction Network for Motion Forecasting](https://mlsanity.com/paper/2311.04512/) (2023)
- [Neural Semantic Map-Learning for Autonomous Vehicles](https://mlsanity.com/paper/2410.07780/) (2024)
- [Label Anything: An Interpretable, High-Fidelity and Prompt-Free Annotator](https://mlsanity.com/paper/2502.02972/) (2025)

## Built on by (most cited first, 5 of 5)

- [FedDSR: Federated Deep Supervision and Regularization Towards Autonomous Driving](https://mlsanity.com/paper/2512.06676/) (2025)
- [FedEMA-Distill: Exponential Moving Average Guided Knowledge Distillation for Robust Federated Learning](https://mlsanity.com/paper/2603.04422/) (2026)
- [OmniISR: A Unified Framework for Centralized and Federated Learning via Intermediate Supervision and Regularization](https://mlsanity.com/paper/2605.20276/) (2026)
- [AirMoE: Statistic-Augmented Over-the-Air MoE for Collaborative Intelligence](https://mlsanity.com/paper/2607.16562/) (2026)
- [FeDepth: Federated Learning for Depth Estimation under Robot Heterogeneity](https://mlsanity.com/paper/2608.01129/) (2026)

## Abstract (arXiv)

Street Scene Semantic Understanding (denoted as S3U) is a crucial but complex task for autonomous driving (AD) vehicles. Their inference models typically face poor generalization due to domain-shift. Federated Learning (FL) has emerged as a promising paradigm for enhancing the generalization of AD models through privacy-preserving distributed learning. However, these FL AD models face significant temporal catastrophic forgetting when deployed in dynamically evolving environments, where continuous adaptation causes abrupt erosion of historical knowledge. This paper proposes Federated Exponential Moving Average (FedEMA), a novel framework that addresses this challenge through two integral innovations: (I) Server-side model's historical fitting capability preservation via fusing current FL round's aggregation model and a proposed previous FL round's exponential moving average (EMA) model; (II) Vehicle-side negative entropy regularization to prevent FL models' possible overfitting to EMA-introduced temporal patterns. Above two strategies empower FedEMA a dual-objective optimization that balances model generalization and adaptability. In addition, we conduct theoretical convergence analysis for the proposed FedEMA. Extensive experiments both on Cityscapes dataset and Camvid dataset demonstrate FedEMA's superiority over existing approaches, showing 7.12% higher mean Intersection-over-Union (mIoU).