mlsanityData to 14 Oct 2026

Rethinking Training & Inference for Forecasting: Linking Winner-Take-All back to GMMs

arXiv 2606.26424 ↗First posted 24 Jun 2026cs.LG cs.CV cs.ROcited by 0

Not on the map. Why?

Abstract (arXiv)

Trajectory forecasting for autonomous driving has advanced rapidly, yet representative models often produce uninformative posteriors over forecast modes, causing problems for mode pruning. We trace this to a modeling-training mismatch: forecasters are typically modeled as conditional Gaussian mixture models (GMMs) but trained with a winner-take-all (WTA) loss that assigns each sample to its nearest mode. We argue that this K-means-like hard assignment (one-hot), while preventing mode collapse, is the source of uninformative mode probabilities: it over-segments the trajectory space, ignores relatedness among nearby modes, and yields assignment instability under small perturbations. Guided by this lens, we introduce two post-hoc treatments: (1) test-time posterior-weighted merging that aggregates nearby candidate trajectories; and (2) a one-step expectation-maximization (EM) update that replaces hard labels with soft responsibilities, sharing probability mass across neighboring modes. Across several WTA-trained architectures, these lightweight steps produce more informative, faithfully ranked mode posteriors and strengthen final forecasts on popular displacement metrics -- without retraining. Our analysis unifies recent design choices through a GMM-vs-K-means perspective and offers principled, practical corrections that better align training objectives with inference.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 36 references that are in the map. First posted on arXiv 24 Jun 2026.

Builds on

25 of 36 references.

References · 25

  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  2. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 707
  3. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 690
  4. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 537
  5. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 420
  6. 2018SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraintscited by 298
  7. 2022Wayformer: Motion Forecasting via Simple & Efficient Attention Networkscited by 280
  8. 2022Motion Transformer with Global Intention Localization and Local Movement Refinementcited by 270
  9. 2021MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Predictioncited by 246
  10. 2018Convolutional Social Pooling for Vehicle Trajectory Predictioncited by 221
  11. 2019PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settingscited by 179
  12. 2023MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Queryingcited by 140
13 more
  1. 2020Implicit Latent Variable Model for Scene-Consistent Motion Forecastingcited by 116
  2. 2019Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Datacited by 104
  3. 2021LookOut: Diverse Multi-Future Prediction and Planning for Self-Drivingcited by 96
  4. 2018Sequence-to-Sequence Prediction of Vehicle Trajectory via LSTM Encoder-Decoder Architecturecited by 55
  5. 2024UniTraj: A Unified Framework for Scalable Vehicle Trajectory Predictioncited by 52
  6. 2022JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Drivingcited by 40
  7. 2017Situation Recognition with Graph Neural Networkscited by 34
  8. 2019Discrete Residual Flow for Probabilistic Pedestrian Behavior Predictioncited by 34
  9. 2018Probabilistic Prediction of Vehicle Semantic Intention and Motioncited by 31
  10. 2025Scaling Laws of Motion Forecasting and Planning -- Technical Reportcited by 19
  11. 2022MTR-A: 1st Place Solution for 2022 Waymo Open Dataset Challenge -- Motion Predictioncited by 17
  12. 2024Efficient Motion Prediction: A Lightweight & Accurate Trajectory Prediction Model With Fast Training and Inference Speedcited by 10
  13. 2020Multimodal Trajectory Prediction via Topological Invariance for Navigation at Uncontrolled Intersectionscited by 6

Built on by

Not cited yet.