mlsanityData to 7 Oct 2026

FedBEVT: Federated Learning Bird's Eye View Perception Transformer in Road Traffic Systems

arXiv 2304.01534 ↗First posted 4 Apr 2023cs.CVcited by 14

Not on the map. Why?

Abstract (arXiv)

Bird's eye view (BEV) perception is becoming increasingly important in the field of autonomous driving. It uses multi-view camera data to learn a transformer model that directly projects the perception of the road environment onto the BEV perspective. However, training a transformer model often requires a large amount of data, and as camera data for road traffic are often private, they are typically not shared. Federated learning offers a solution that enables clients to collaborate and train models without exchanging data but model parameters. In this paper, we introduce FedBEVT, a federated transformer learning approach for BEV perception. In order to address two common data heterogeneity issues in FedBEVT: (i) diverse sensor poses, and (ii) varying sensor numbers in perception systems, we propose two approaches -- Federated Learning with Camera-Attentive Personalization (FedCaP) and Adaptive Multi-Camera Masking (AMCM), respectively. To evaluate our method in real-world settings, we create a dataset consisting of four typical federated use cases. Our findings suggest that FedBEVT outperforms the baseline approaches in all four use cases, demonstrating the potential of our approach for improving BEV perception in autonomous driving.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 55 of its 90 references that are in the map. First posted on arXiv 4 Apr 2023.

Builds on

55 of 90 references.

References · 55

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  6. 2019Advances and Open Problems in Federated Learningcited by 3,021
  7. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  8. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  9. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  10. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  11. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  12. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
38 more
  1. 2018LEAF: A Benchmark for Federated Settingscited by 780
  2. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
  3. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  4. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 550
  5. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  6. 2019Federated Learning with Personalization Layerscited by 533
  7. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 471
  8. 2021Towards Personalized Federated Learningcited by 462
  9. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 351
  10. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 322
  11. 2022V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformercited by 320
  12. 2021OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communicationcited by 304
  13. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 282
  14. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 264
  15. 2022Cross-view Transformers for real-time Map-view Semantic Segmentationcited by 264
  16. 2022CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformerscited by 227
  17. 2021Personalized Federated Learning using Hypernetworkscited by 210
  18. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  19. 2021FedKD: Communication Efficient Federated Learning via Knowledge Distillationcited by 185
  20. 2023V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perceptioncited by 155
  21. 2022Vision-Centric BEV Perception: A Surveycited by 126
  22. 2022Federated Learning with Partial Model Personalizationcited by 114
  23. 2021Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learningcited by 111
  24. 2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 108
  25. 2023Street-View Image Generation from a Bird's-Eye View Layoutcited by 102
  26. 2023FedGH: Heterogeneous Federated Learning with Generalized Global Headercited by 56
  27. 2022Communication-Efficient Adaptive Federated Learningcited by 55
  28. 2021Deep Federated Learning for Autonomous Drivingcited by 54
  29. 2020Federated Learning in Vehicular Networkscited by 53
  30. 2022FedDrive: Generalizing Federated Learning to Semantic Segmentation in Autonomous Drivingcited by 41
  31. 2022FedTP: Federated Learning by Transformer Personalizationcited by 36
  32. 2021Personalized Federated Learning: A Unified Framework and Universal Optimization Techniquescited by 34
  33. 2019Federated User Representation Learningcited by 31
  34. 2020The OARF Benchmark Suite: Characterization and Implications for Federated Learning Systemscited by 31
  35. 2019Real-World Image Datasets for Federated Learningcited by 30
  36. 2023The OpenCDA Open-source Ecosystem for Cooperative Driving Automation Researchcited by 27
  37. 2019Federated Imitation Learning: A Novel Framework for Cloud Robotic Systems with Heterogeneous Sensor Datacited by 24
  38. 2021Real-time End-to-End Federated Learning: An Automotive Case Studycited by 15

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 6; 2025: 4; 2026: 1.

14 papers.

Sort:

  1. 2024A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlookcited by 65
  2. 2023Data-driven Traffic Simulation: A Comprehensive Reviewcited by 29
  3. 2023CoBEV: Elevating Roadside 3D Object Detection with Depth and Height Complementaritycited by 18
  4. 2024Importance-Aware Adaptive Dataset Distillationcited by 18
  5. 2023Vehicle-to-Everything Cooperative Perception for Autonomous Drivingcited by 15
  6. 2024A Survey on Intermediate Fusion Methods for Collaborative Perception Categorized by Real World Challengescited by 13
  7. 2024OE-BevSeg: An Object Informed and Environment Aware Multimodal Framework for Bird's-eye-view Vehicle Semantic Segmentationcited by 3
  8. 2024FedRSU: Federated Learning for Scene Flow Estimation on Roadside Unitscited by 2
  9. 2024On the Federated Learning Framework for Cooperative Perceptioncited by 2
  10. 2025Towards One-shot Federated Learning: Advances, Challenges, and Future Directionscited by 2
  11. 2025RG-Attn: Radian Glue Attention for Multi-modality Multi-agent Cooperative Perceptioncited by 1
  12. 2025Enabling Federated Object Detection for Connected Autonomous Vehicles: A Deployment-Oriented Evaluationcited by 0
2 more
  1. 2025DRCP: Diffusion on Reinforced Cooperative Perception for Perceiving Beyond Limitscited by 0
  2. 2026SimBEV2X: A Large-Scale Dataset and Data Generation Tool for Multi-Task Vehicle-to-Everything Cooperative Perceptioncited by 0