# Bayesian Prediction of Future Street Scenes using Synthetic Likelihoods

arXiv: https://arxiv.org/abs/1810.00746 · HTML: https://mlsanity.com/paper/1810.00746/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Oct 2018.

## Built on by (most cited first, 17 of 17)

- [DINO-Foresight: Looking into the Future with DINO](https://mlsanity.com/paper/2412.11673/) (2024)
- [Real-time Object Detection for Streaming Perception](https://mlsanity.com/paper/2203.12338/) (2022)
- [Rethinking Learning Approaches for Long-Term Action Anticipation](https://mlsanity.com/paper/2210.11566/) (2022)
- [Revisiting Hierarchical Approach for Persistent Long-Term Video Prediction](https://mlsanity.com/paper/2104.06697/) (2021)
- [Multi-Modal Temporal Convolutional Network for Anticipating Actions in Egocentric Videos](https://mlsanity.com/paper/2107.09504/) (2021)
- [Learning Semantic-Aware Dynamics for Video Prediction](https://mlsanity.com/paper/2104.09762/) (2021)
- [Panoptic Segmentation Forecasting](https://mlsanity.com/paper/2104.03962/) (2021)
- [StreamYOLO: Real-time Object Detection for Streaming Perception](https://mlsanity.com/paper/2207.10433/) (2022)
- [Frozen Forecasting: A Unified Evaluation](https://mlsanity.com/paper/2507.13942/) (2025)
- [Dense Semantic Forecasting in Video by Joint Regression of Features and Feature Motion](https://mlsanity.com/paper/2101.10777/) (2021)
- [Joint Forecasting of Panoptic Segmentations with Difference Attention](https://mlsanity.com/paper/2204.07157/) (2022)
- [Forecasting of depth and ego-motion with transformers and self-supervision](https://mlsanity.com/paper/2206.07435/) (2022)
- [DaDe: Delay-adaptive Detector for Streaming Perception](https://mlsanity.com/paper/2212.11558/) (2022)
- [Long-Term Prediction of Natural Video Sequences with Robust Video Predictors](https://mlsanity.com/paper/2308.11079/) (2023)
- [Borrowing from yourself: Faster future video segmentation with partial channel update](https://mlsanity.com/paper/2202.05748/) (2022)
- [On depth prediction for autonomous driving using self-supervised learning](https://mlsanity.com/paper/2403.06194/) (2024)
- [CEC-MMR: Cross-Entropy Clustering Approach to Multi-Modal Regression](https://mlsanity.com/paper/2504.07301/) (2025)

## Abstract (arXiv)

For autonomous agents to successfully operate in the real world, the ability to anticipate future scene states is a key competence. In real-world scenarios, future states become increasingly uncertain and multi-modal, particularly on long time horizons. Dropout based Bayesian inference provides a computationally tractable, theoretically well grounded approach to learn likely hypotheses/models to deal with uncertain futures and make predictions that correspond well to observations -- are well calibrated. However, it turns out that such approaches fall short to capture complex real-world scenes, even falling behind in accuracy when compared to the plain deterministic approaches. This is because the used log-likelihood estimate discourages diversity. In this work, we propose a novel Bayesian formulation for anticipating future scene states which leverages synthetic likelihoods that encourage the learning of diverse models to accurately capture the multi-modal nature of future scene states. We show that our approach achieves accurate state-of-the-art predictions and calibrated probabilities through extensive experiments for scene anticipation on Cityscapes dataset. Moreover, we show that our approach generalizes across diverse tasks such as digit generation and precipitation forecasting.