mlsanityData to 13 Oct 2026

TurboTrain: Towards Efficient and Balanced Multi-Task Learning for Multi-Agent Perception and Prediction

arXiv 2508.04682 ↗First posted 6 Aug 2025cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

End-to-end training of multi-agent systems offers significant advantages in improving multi-task performance. However, training such models remains challenging and requires extensive manual design and monitoring. In this work, we introduce TurboTrain, a novel and efficient training framework for multi-agent perception and prediction. TurboTrain comprises two key components: a multi-agent spatiotemporal pretraining scheme based on masked reconstruction learning and a balanced multi-task learning strategy based on gradient conflict suppression. By streamlining the training process, our framework eliminates the need for manually designing and tuning complex multi-stage training pipelines, substantially reducing training time and improving performance. We evaluate TurboTrain on a real-world cooperative driving dataset, V2XPnP-Seq, and demonstrate that it further improves the performance of state-of-the-art multi-agent perception and prediction models. Our results highlight that pretraining effectively captures spatiotemporal multi-agent features and significantly benefits downstream tasks. Moreover, the proposed balanced multi-task learning strategy enhances detection and prediction.

Key facts. Cited by 9 papers in the map (data through 13 Oct 2026). Builds on 67 of its 87 references that are in the map. First posted on arXiv 6 Aug 2025.

Builds on

67 of 87 references.

References · 67

  1. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,364
  2. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,709
  3. 2020Gradient Surgery for Multi-Task Learningcited by 1,225
  4. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 822
  5. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 615
  6. 2019RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Cloudscited by 585
  7. 2018End-to-End Multi-Task Learning with Attentioncited by 533
  8. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 446
  9. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428
  10. 2022V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformercited by 320
  11. 2021OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communicationcited by 304
  12. 2020V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Predictioncited by 292
38 more
  1. 2022Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Mapscited by 255
  2. 2024SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representationcited by 240
  3. 2020PointASNL: Robust Point Clouds Processing using Nonlocal Neural Networks with Adaptive Samplingcited by 238
  4. 2021Learning Distilled Collaboration Graph for Multi-Agent Perceptioncited by 194
  5. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  6. 2019F-Cooper: Feature based Cooperative Perception for Autonomous Vehicle Edge Computing System Using 3D Point Cloudscited by 171
  7. 2022V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Drivingcited by 157
  8. 2022ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queriescited by 116
  9. 2023V2X-Seq: A Large-Scale Sequential Dataset for Vehicle-Infrastructure Cooperative Perception and Forecastingcited by 114
  10. 2021Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learningcited by 101
  11. 2024An Extensible Framework for Open Heterogeneous Collaborative Perceptioncited by 87
  12. 2020LiDAR-based Panoptic Segmentation via Dynamic Shifting Networkcited by 84
  13. 2023UniPAD: A Universal Pre-training Paradigm for Autonomous Drivingcited by 73
  14. 2024TUMTraf V2X Cooperative Perception Datasetcited by 66
  15. 2024V2X-Real: a Large-Scale Dataset for Vehicle-to-Everything Cooperative Perceptioncited by 65
  16. 2023VMT-Adapter: Parameter-Efficient Transfer Learning for Multi-Task Dense Scene Understandingcited by 62
  17. 2020Random Hypervolume Scalarizations for Provable Multi-Objective Black Box Optimizationcited by 60
  18. 2022Occupancy-MAE: Self-supervised Pre-training Large-scale LiDAR Point Clouds with Masked Occupancy Autoencoderscited by 58
  19. 2023Asynchrony-Robust Collaborative Perception via Bird's Eye View Flowcited by 58
  20. 2022GD-MAE: Generative Decoder for MAE Pre-training on LiDAR Point Cloudscited by 57
  21. 2023Spatio-Temporal Domain Awareness for Multi-Agent Collaborative Perceptioncited by 52
  22. 2023Revisiting Scalarization in Multi-Task Learning: A Theoretical Perspectivecited by 51
  23. 2019MultiNet++: Multi-Stream Feature Aggregation and Geometric Loss Strategy for Multi-Task Learningcited by 45
  24. 2025STAMP: Scalable Task And Model-agnostic Collaborative Perceptioncited by 44
  25. 2023Traj-MAE: Masked Autoencoders for Trajectory Predictioncited by 42
  26. 2024End-to-End Autonomous Driving through V2X Cooperationcited by 42
  27. 2023CORE: Cooperative Reconstruction for Multi-Agent Perceptioncited by 41
  28. 2024UnO: Unsupervised Occupancy Fields for Perception and Forecastingcited by 40
  29. 2023GeoMAE: Masked Geometric Target Prediction for Self-supervised Point Cloud Pre-Trainingcited by 38
  30. 2023Towards Vehicle-to-everything Autonomous Driving: A Survey on Collaborative Perceptioncited by 38
  31. 2024Collaborative Semantic Occupancy Prediction with Hybrid Feature Fusion in Connected Automated Vehiclescited by 33
  32. 20244D Contrastive Superflows are Dense 3D Representation Learnerscited by 32
  33. 2023Flow-Based Feature Fusion for Vehicle-Infrastructure Cooperative 3D Object Detectioncited by 31
  34. 2024V2XPnP: Vehicle-to-Everything Spatio-Temporal Fusion for Multi-Agent Perception and Predictioncited by 31
  35. 2022Effective Adaptation in Multi-Task Co-Training for Unified Autonomous Drivingcited by 30
  36. 2023Implicit Occupancy Flow Fields for Perception and Prediction in Self-Drivingcited by 30
  37. 2022PreTraM: Self-Supervised Pre-training via Connecting Trajectory and Mapcited by 25
  38. 2024CMP: Cooperative Motion Prediction with Multi-Agent Communicationcited by 25

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 3; 2026: 6.

9 papers.

  1. 2026SpanVLA: Efficient Action Bridging and Learning from Negative-Recovery Samples for Vision-Language-Action Modelcited by 17
  2. 2025QuantV2X: A Fully Quantized Multi-Agent System for Cooperative Perceptioncited by 11
  3. 2026MDrive: Benchmarking Closed-Loop Cooperative Driving for End-to-End Multi-agent Systemscited by 5
  4. 2025Forging Spatial Intelligence: A Roadmap of Multi-Modal Data Pre-Training for Autonomous Systemscited by 4
  5. 2025SafeCoop: Unravelling Full Stack Safety in Agentic Collaborative Drivingcited by 2
  6. 2026CMU-Drive and V2V-VLA: Cooperative Multi-agent Unified Driving with Reasoning Benchmark and Vehicle-to-Vehicle Vision-Language-Action Modelscited by 1
  7. 2026When the City Teaches the Car: Label-Free 3D Perception from Infrastructurecited by 0
  8. 2026ConFixGS: Learning to Fix Feedforward 3D Gaussian Splatting with Confidence-Aware Diffusion Priors in Driving Scenescited by 0
  9. 2026VIPS: Vehicle-Infrastructure Cooperative Planning Benchmark via Pseudo-Simulationcited by 0