# ViewBirdiformer: Learning to recover ground-plane crowd trajectories and ego-motion from a single ego-centric view

arXiv: https://arxiv.org/abs/2210.06332 · HTML: https://mlsanity.com/paper/2210.06332/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 40 references that are in the map.
- First posted on arXiv 12 Oct 2022.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [MOT16: A Benchmark for Multi-Object Tracking](https://mlsanity.com/paper/1603.00831/) (2016)
- [AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting](https://mlsanity.com/paper/2103.14023/) (2021)
- [Cross-view Transformers for real-time Map-view Semantic Segmentation](https://mlsanity.com/paper/2205.02833/) (2022)
- [DynaSLAM: Tracking, Mapping and Inpainting in Dynamic Scenes](https://mlsanity.com/paper/1806.05620/) (2018)
- [Translating Images into Maps](https://mlsanity.com/paper/2110.00966/) (2021)
- [GNN3DMOT: Graph Neural Network for 3D Multi-Object Tracking with Multi-Feature Learning](https://mlsanity.com/paper/2006.07327/) (2020)
- [CubeSLAM: Monocular 3D Object SLAM](https://mlsanity.com/paper/1806.00557/) (2018)
- [DS-SLAM: A Semantic Visual SLAM towards Dynamic Environments](https://mlsanity.com/paper/1809.08379/) (2018)
- [Monocular Quasi-Dense 3D Object Tracking](https://mlsanity.com/paper/2103.07351/) (2021)
- [What the Constant Velocity Model Can Teach Us About Pedestrian Motion Prediction](https://mlsanity.com/paper/1903.07933/) (2019)
- [Joint Monocular 3D Vehicle Detection and Tracking](https://mlsanity.com/paper/1811.10742/) (2018)
- [Track to Reconstruct and Reconstruct to Track](https://mlsanity.com/paper/1910.00130/) (2019)
- [Camera Calibration and Player Localization in SoccerNet-v2 and Investigation of their Representations for Action Spotting](https://mlsanity.com/paper/2104.09333/) (2021)
- [Beyond Pixels: Leveraging Geometry and Shape Cues for Online Multi-Object Tracking](https://mlsanity.com/paper/1802.09298/) (2018)
- [MonoLoco: Monocular 3D Pedestrian Localization and Uncertainty Estimation](https://mlsanity.com/paper/1906.06059/) (2019)
- [Differentiable SLAM-net: Learning Particle SLAM for Visual Navigation](https://mlsanity.com/paper/2105.07593/) (2021)
- [Combined Image- and World-Space Tracking in Traffic Scenes](https://mlsanity.com/paper/1809.07357/) (2018)
- [Dynamic SLAM: The Need For Speed](https://mlsanity.com/paper/2002.08584/) (2020)
- [ClusterVO: Clustering Moving Instances and Estimating Visual Odometry for Self and Surroundings](https://mlsanity.com/paper/2003.12980/) (2020)
- [PillarFlow: End-to-end Birds-eye-view Flow Estimation for Autonomous Driving](https://mlsanity.com/paper/2008.01179/) (2020)
- [DOT: Dynamic Object Tracking for Visual SLAM](https://mlsanity.com/paper/2010.00052/) (2020)
- [Dynamic Object Tracking and Masking for Visual SLAM](https://mlsanity.com/paper/2008.00072/) (2020)
- [L2B: Learning to Balance the Safety-Efficiency Trade-off in Interactive Crowd-aware Robot Navigation](https://mlsanity.com/paper/2003.09207/) (2020)
- [View Birdification in the Crowd: Ground-Plane Localization from Perceived Movements](https://mlsanity.com/paper/2111.05060/) (2021)

## Built on by (most cited first, 3 of 3)

- [Physical Plausibility-aware Trajectory Prediction via Locomotion Embodiment](https://mlsanity.com/paper/2503.17267/) (2025)
- [Bridging Perspectives: A Survey on Cross-view Collaborative Intelligence with Egocentric-Exocentric Vision](https://mlsanity.com/paper/2506.06253/) (2025)
- [Locking On: Leveraging Dynamic Vehicle-Imposed Motion Constraints to Improve Visual Localization](https://mlsanity.com/paper/2306.17529/) (2023)

## Abstract (arXiv)

We introduce a novel learning-based method for view birdification, the task of recovering ground-plane trajectories of pedestrians of a crowd and their observer in the same crowd just from the observed ego-centric video. View birdification becomes essential for mobile robot navigation and localization in dense crowds where the static background is hard to see and reliably track. It is challenging mainly for two reasons; i) absolute trajectories of pedestrians are entangled with the movement of the observer which needs to be decoupled from their observed relative movements in the ego-centric video, and ii) a crowd motion model describing the pedestrian movement interactions is specific to the scene yet unknown a priori. For this, we introduce a Transformer-based network referred to as ViewBirdiformer which implicitly models the crowd motion through self-attention and decomposes relative 2D movement observations onto the ground-plane trajectories of the crowd and the camera through cross-attention between views. Most important, ViewBirdiformer achieves view birdification in a single forward pass which opens the door to accurate real-time, always-on situational awareness. Extensive experimental results demonstrate that ViewBirdiformer achieves accuracy similar to or better than state-of-the-art with three orders of magnitude reduction in execution time.