# Pictura: Perspective-View Self-Play at Scale for Driving

arXiv: https://arxiv.org/abs/2607.26005 · HTML: https://mlsanity.com/paper/2607.26005/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving.
- Builds on 31 of its 39 references that are in the map.
- First posted on arXiv 28 Jul 2026.

## Placed in

- End-to-end autonomous driving / [Imitation-learning end-to-end planners](https://mlsanity.com/driving/imitation-learning-end-to-end-planners/)

## Builds on (most cited first, 30 of 31)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022)
- [Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset](https://mlsanity.com/paper/2104.10133/) (2021)
- [NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles](https://mlsanity.com/paper/2106.11810/) (2021)
- [Learning by Cheating](https://mlsanity.com/paper/1912.12294/) (2019)
- [MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning](https://mlsanity.com/paper/2109.12674/) (2021)
- [A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning](https://mlsanity.com/paper/1711.00832/) (2017)
- [End-to-End Urban Driving by Imitating a Reinforcement Learning Coach](https://mlsanity.com/paper/2108.08265/) (2021)
- [Panacea: Panoramic and Controllable Video Generation for Autonomous Driving](https://mlsanity.com/paper/2311.16813/) (2023)
- [Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research](https://mlsanity.com/paper/2310.08710/) (2023)
- [DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Driving](https://mlsanity.com/paper/2308.00398/) (2023)
- [Learning to drive from a world on rails](https://mlsanity.com/paper/2105.00636/) (2021)
- [HUGSIM: A Real-Time, Photo-Realistic and Closed-Loop Simulator for Autonomous Driving](https://mlsanity.com/paper/2412.01718/) (2024)
- [Cosmos-Drive-Dreams: Scalable Synthetic Driving Data Generation with World Foundation Models](https://mlsanity.com/paper/2506.09042/) (2025)
- [VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehicles](https://mlsanity.com/paper/2111.12083/) (2021)
- [Robust Autonomy Emerges from Self-Play](https://mlsanity.com/paper/2502.03349/) (2025)
- [NeuroNCAP: Photorealistic Closed-loop Safety Testing for Autonomous Driving](https://mlsanity.com/paper/2404.07762/) (2024)
- [GPUDrive: Data-driven, multi-agent driving simulation at 1 million FPS](https://mlsanity.com/paper/2408.01584/) (2024)
- [ReSim: Reliable World Simulation for Autonomous Driving](https://mlsanity.com/paper/2506.09981/) (2025)
- [Driving on Registers](https://mlsanity.com/paper/2601.05083/) (2026)
- [RAP: 3D Rasterization Augmented End-to-End Planning](https://mlsanity.com/paper/2510.04333/) (2025)
- [PufferLib: Making Reinforcement Learning Libraries and Environments Play Nice](https://mlsanity.com/paper/2406.12905/) (2024)
- [LEAD: Minimizing Learner-Expert Asymmetry in End-to-End Driving](https://mlsanity.com/paper/2512.20563/) (2025)
- [Towards Learning Multi-agent Negotiations via Self-Play](https://mlsanity.com/paper/2001.10208/) (2020)
- [Learning to Drive via Asymmetric Self-Play](https://mlsanity.com/paper/2409.18218/) (2024)
- [MAD: Motion Appearance Decoupling for efficient Driving World Models](https://mlsanity.com/paper/2601.09452/) (2026)
- [Human-like autonomy emerges from self-play and a pinch of human data](https://mlsanity.com/paper/2606.19370/) (2026)
- [Scaling Self-Play for End-to-End Driving](https://mlsanity.com/paper/2606.19641/) (2026)

## Built on by (most cited first, 2 of 2)

- [DriveZero: End-to-End Driving Beyond Human Demonstrations](https://mlsanity.com/paper/2609.06055/) (2026)
- [OPTED: On-Policy Fine-Tuning for End-to-End Driving using a Render-Free Teacher](https://mlsanity.com/paper/2609.20756/) (2026)

## Abstract (arXiv)

Self-play in simulation produces robust driving policies at scale. Demonstrations of such behavior have been made using privileged vectorized observations such as exact poses and velocities, even for occluded agents. This assumes that perception is solved and introduces a representation gap with the partial observation of a deployed agent driving from the perspective view of egocentric cameras. A common fix, distilling the privileged policy into a camera-input student, leaves the student imitating decisions its own view cannot justify. Instead, we establish perspective-view self-play as a practical training regime. We introduce Pictura, a GPU-accelerated multi-agent driving simulator that renders each agent's egocentric view at every step, mitigating the representation gap at its source. Pictura sustains up to 500K agent-steps/s (2M images/s) on a single H100. Using Pictura, we train Alberti by self-play with plain PPO. It is the first large-scale driving self-play policy trained directly from perspective images, without privileged observations. Training spans 50B agent steps for ~35M km of driving. It approaches the driving performance of its privileged vectorized counterpart, and transfers zero-shot to Waymo Open Motion Dataset layouts re-rendered in Pictura, where it outperforms privileged vectorized agents. Project page: https://valeoai.github.io/Pictura/