# Single Shot Multitask Pedestrian Detection and Behavior Prediction

arXiv: https://arxiv.org/abs/2101.02232 · HTML: https://mlsanity.com/paper/2101.02232/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 24 references that are in the map.
- First posted on arXiv 6 Jan 2021.

## Builds on (most cited first, 16 of 16)

- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [Cross-stitch Networks for Multi-task Learning](https://mlsanity.com/paper/1604.03539/) (2016)
- [Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net](https://mlsanity.com/paper/2012.12395/) (2020)
- [PnPNet: End-to-End Perception and Prediction with Tracking in the Loop](https://mlsanity.com/paper/2005.14711/) (2020)
- [Analyzing the Variety Loss in the Context of Probabilistic Trajectory Prediction](https://mlsanity.com/paper/1907.10178/) (2019)
- [RNN-based Pedestrian Crossing Prediction using Activity and Pose-related Features](https://mlsanity.com/paper/2008.11647/) (2020)
- [Intention Recognition of Pedestrians and Cyclists by 2D Pose Estimation](https://mlsanity.com/paper/1910.03858/) (2019)
- [Pedestrian Models for Autonomous Driving Part I: Low-Level Models, from Sensing to Tracking](https://mlsanity.com/paper/2002.11669/) (2020)
- [Generalizable Pedestrian Detection: The Elephant In The Room](https://mlsanity.com/paper/2003.08799/) (2020)
- [VRUNet: Multi-Task Learning Model for Intent Prediction of Vulnerable Road Users](https://mlsanity.com/paper/2007.05397/) (2020)

## Built on by (most cited first, 1 of 1)

- [Semantic and Articulated Pedestrian Sensing Onboard a Moving Vehicle](https://mlsanity.com/paper/2309.06313/) (2023)

## Abstract (arXiv)

Detecting and predicting the behavior of pedestrians is extremely crucial for self-driving vehicles to plan and interact with them safely. Although there have been several research works in this area, it is important to have fast and memory efficient models such that it can operate in embedded hardware in these autonomous machines. In this work, we propose a novel architecture using spatial-temporal multi-tasking to do camera based pedestrian detection and intention prediction. Our approach significantly reduces the latency by being able to detect and predict all pedestrians' intention in a single shot manner while also being able to attain better accuracy by sharing features with relevant object level information and interactions.