# INFER: INtermediate representations for FuturE pRediction

arXiv: https://arxiv.org/abs/1903.10641 · HTML: https://mlsanity.com/paper/1903.10641/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Mar 2019.

## Built on by (most cited first, 12 of 12)

- [Explainability of deep vision-based autonomous driving systems: Review and challenges](https://mlsanity.com/paper/2101.05307/) (2021)
- [On Exposing the Challenging Long Tail in Future Prediction of Traffic Actors](https://mlsanity.com/paper/2103.12474/) (2021)
- [Divide-and-Conquer for Lane-Aware Diverse Trajectory Prediction](https://mlsanity.com/paper/2104.08277/) (2021)
- [LaRa: Latents and Rays for Multi-Camera Bird's-Eye-View Semantic Segmentation](https://mlsanity.com/paper/2206.13294/) (2022)
- [SMEMO: Social Memory for Trajectory Forecasting](https://mlsanity.com/paper/2203.12446/) (2022)
- [CausalCity: Complex Simulations with Agency for Causal Discovery and Reasoning](https://mlsanity.com/paper/2106.13364/) (2021)
- [Congestion-aware Multi-agent Trajectory Prediction for Collision Avoidance](https://mlsanity.com/paper/2103.14231/) (2021)
- [Unlimited Neighborhood Interaction for Heterogeneous Trajectory Prediction](https://mlsanity.com/paper/2108.00238/) (2021)
- [LiDARTouch: Monocular metric depth estimation with a few-beam LiDAR](https://mlsanity.com/paper/2109.03569/) (2021)
- [Raising context awareness in motion forecasting](https://mlsanity.com/paper/2109.08048/) (2021)
- [Exploiting latent representation of sparse semantic layers for improved short-term motion prediction with Capsule Networks](https://mlsanity.com/paper/2103.01644/) (2021)
- [Multi-modal anticipation of stochastic trajectories in a dynamic environment with Conditional Variational Autoencoders](https://mlsanity.com/paper/2103.03912/) (2021)

## Abstract (arXiv)

In urban driving scenarios, forecasting future trajectories of surrounding vehicles is of paramount importance. While several approaches for the problem have been proposed, the best-performing ones tend to require extremely detailed input representations (eg. image sequences). But, such methods do not generalize to datasets they have not been trained on. We propose intermediate representations that are particularly well-suited for future prediction. As opposed to using texture (color) information, we rely on semantics and train an autoregressive model to accurately predict future trajectories of traffic participants (vehicles) (see fig. above). We demonstrate that using semantics provides a significant boost over techniques that operate over raw pixel intensities/disparities. Uncharacteristic of state-of-the-art approaches, our representations and models generalize to completely different datasets, collected across several cities, and also across countries where people drive on opposite sides of the road (left-handed vs right-handed driving). Additionally, we demonstrate an application of our approach in multi-object tracking (data association). To foster further research in transferrable representations and ensure reproducibility, we release all our code and data.