# Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction

arXiv: https://arxiv.org/abs/1906.03631 · HTML: https://mlsanity.com/paper/1906.03631/

## Key facts

- Cited by 96 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jun 2019.

## Built on by (most cited first, 30 of 96)

- [Stochastic Scene-Aware Motion Prediction](https://mlsanity.com/paper/2108.08284/) (2021)
- [Multimodal Trajectory Prediction Conditioned on Lane-Graph Traversals](https://mlsanity.com/paper/2106.15004/) (2021)
- [SAGA: Stochastic Whole-Body Grasping with Contact](https://mlsanity.com/paper/2112.10103/) (2021)
- [EPro-PnP: Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose Estimation](https://mlsanity.com/paper/2303.12787/) (2023)
- [LAformer: Trajectory Prediction for Autonomous Driving with Lane-Aware Scene Constraints](https://mlsanity.com/paper/2302.13933/) (2023)
- [Continuous 3D Multi-Channel Sign Language Production via Progressive Transformers and Mixture Density Networks](https://mlsanity.com/paper/2103.06982/) (2021)
- [On Exposing the Challenging Long Tail in Future Prediction of Traffic Actors](https://mlsanity.com/paper/2103.12474/) (2021)
- [Task-Oriented Human-Object Interactions Generation with Implicit Neural Representations](https://mlsanity.com/paper/2303.13129/) (2023)
- [Closed-Loop Supervised Fine-Tuning of Tokenized Traffic Models](https://mlsanity.com/paper/2412.05334/) (2024)
- [Divide-and-Conquer for Lane-Aware Diverse Trajectory Prediction](https://mlsanity.com/paper/2104.08277/) (2021)
- [FEND: A Future Enhanced Distribution-Aware Contrastive Learning Framework for Long-tail Trajectory Prediction](https://mlsanity.com/paper/2303.16574/) (2023)
- [GATraj: A Graph- and Attention-based Multi-Agent Trajectory Prediction Model](https://mlsanity.com/paper/2209.07857/) (2022)
- [SMART: Scalable Multi-agent Real-time Motion Generation via Next-token Prediction](https://mlsanity.com/paper/2405.15677/) (2024)
- [HuManiFlow: Ancestor-Conditioned Normalising Flows on SO(3) Manifolds for Human Pose and Shape Distribution Estimation](https://mlsanity.com/paper/2305.06968/) (2023)
- [You Mostly Walk Alone: Analyzing Feature Attribution in Trajectory Prediction](https://mlsanity.com/paper/2110.05304/) (2021)
- [Bootstrap Motion Forecasting With Self-Consistent Constraints](https://mlsanity.com/paper/2204.05859/) (2022)
- [End-to-End Trajectory Distribution Prediction Based on Occupancy Grid Maps](https://mlsanity.com/paper/2203.16910/) (2022)
- [Back to the Manifold: Recovering from Out-of-Distribution States](https://mlsanity.com/paper/2207.08673/) (2022)
- [Heterogeneous-Agent Trajectory Forecasting Incorporating Class Uncertainty](https://mlsanity.com/paper/2104.12446/) (2021)
- [MTP-GO: Graph-Based Probabilistic Multi-Agent Trajectory Prediction with Neural ODEs](https://mlsanity.com/paper/2302.00735/) (2023)
- [Injecting Knowledge in Data-driven Vehicle Trajectory Predictors](https://mlsanity.com/paper/2103.04854/) (2021)
- [Distributional Depth-Based Estimation of Object Articulation Models](https://mlsanity.com/paper/2108.05875/) (2021)
- [Unlimited Neighborhood Interaction for Heterogeneous Trajectory Prediction](https://mlsanity.com/paper/2108.00238/) (2021)
- [RSPT: Reconstruct Surroundings and Predict Trajectories for Generalizable Active Object Tracking](https://mlsanity.com/paper/2304.03623/) (2023)
- [Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directions](https://mlsanity.com/paper/2503.03262/) (2025)
- [AVGCN: Trajectory Prediction using Graph Convolutional Networks Guided by Human Attention](https://mlsanity.com/paper/2101.05682/) (2021)
- [Decoder Fusion RNN: Context and Interaction Aware Decoders for Trajectory Prediction](https://mlsanity.com/paper/2108.05814/) (2021)
- [HYPER: Learned Hybrid Trajectory Prediction via Factored Inference and Adaptive Sampling](https://mlsanity.com/paper/2110.02344/) (2021)
- [Diverse Multiple Trajectory Prediction Using a Two-stage Prediction Network Trained with Lane Loss](https://mlsanity.com/paper/2206.08641/) (2022)
- [Safety-compliant Generative Adversarial Networks for Human Trajectory Forecasting](https://mlsanity.com/paper/2209.12243/) (2022)

## Abstract (arXiv)

Future prediction is a fundamental principle of intelligence that helps plan actions and avoid possible dangers. As the future is uncertain to a large extent, modeling the uncertainty and multimodality of the future states is of great relevance. Existing approaches are rather limited in this regard and mostly yield a single hypothesis of the future or, at the best, strongly constrained mixture components that suffer from instabilities in training and mode collapse. In this work, we present an approach that involves the prediction of several samples of the future with a winner-takes-all loss and iterative grouping of samples to multiple modes. Moreover, we discuss how to evaluate predicted multimodal distributions, including the common real scenario, where only a single sample from the ground-truth distribution is available for evaluation. We show on synthetic and real data that the proposed approach triggers good estimates of multimodal distributions and avoids mode collapse. Source code is available at hrefhttps://github.com/lmb-freiburg/Multimodal-Future-Predictionthis https URL.