S2Planner: Multi-Scale Semantic Planner for End-to-End Autonomous Driving
On the map: End-to-end autonomous driving → End-to-end system architecture
Abstract (arXiv)
We present S2Planner, a trajectory planner that combines three front-facing cameras with ego-motion history and the current driving command. A fine-tuned DINOv3 backbone and a Spatial Tuning Adapter produce multi-scale image features; a coarse-to-fine decoder then uses trajectory self-attention and camera-projected cross-attention to refine candidate waypoints. The contribution is the integration of ego-conditioned trajectory initialization with iterative, geometry-guided sampling of multi-scale image features, rather than a new visual backbone or attention operator. On the NAVSIM v1 non-reactive evaluation, the previously reported navtest run obtained 88.03 PDMS. Because that run was selected using navtest performance, this number is exploratory and cannot be interpreted as an unbiased test estimate. Validation-selected evaluation on unexposed data, repeated runs, and computational measurements are needed to establish generalization and efficiency.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in End-to-end system architecture, one of 9 directions in End-to-end autonomous driving. Builds on 42 of its 44 references that are in the map. First posted on arXiv 24 Sep 2026.
Builds on
42 of 44 references.
Closest ancestors · 5
Same direction, newest first
- 2025iPad: Iterative Proposal-centric End-to-End Autonomous Driving21 May
- 2024DRAMA: An Efficient End-to-end Motion Planner for Autonomous Driving with Mamba7 Aug
- 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking21 Jun
- 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving10 May
- 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving31 May
Other references · 37
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2016Feature Pyramid Networks for Object Detectioncited by 5,021
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
- 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
25 more
- 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
- 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
- 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
- 2025DINOv3cited by 1,297
- 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
- 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
- 2022Planning-oriented Autonomous Drivingcited by 1,068
- 2016End to End Learning for Self-Driving Carscited by 1,000
- 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
- 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
- 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 687
- 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 655
- 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 613
- 2022DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRcited by 569
- 2017End-to-end Driving via Conditional Imitation Learningcited by 479
- 2021Conditional DETR for Fast Training Convergencecited by 422
- 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 340
- 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 324
- 2022Vision Transformer with Deformable Attentioncited by 293
- 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 287
- 2024Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillationcited by 244
- 2021YOLOP: You Only Look Once for Panoptic Driving Perceptioncited by 82
- 2024DEIM: DETR with Improved Matching for Fast Convergencecited by 67
- 2025Real-Time Object Detection Meets DINOv3cited by 26
- 2025DiffRefiner: Coarse to Fine Trajectory Planning via Diffusion Refinement with Semantic Interaction for End to End Autonomous Drivingcited by 9
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025MTR-VP: Towards End-to-End Trajectory Planning through Context-Driven Image Encoding and Multiple Trajectory Prediction
- 2025LLaViDA: A Large Language Vision Driving Assistant for Explicit Reasoning and Enhanced Trajectory Planning
- 2025PRIX: Learning to Plan from Raw Pixels for End-to-End Autonomous Driving
- 2025Goal-Based Vision-Language Driving
- 2025HiP-AD: Hierarchical and Multi-Granularity Planning with Deformable Attention for Autonomous Driving in a Single Decoder
- 2025DAP: A Discrete-token Autoregressive Planner for Autonomous Driving
- 2025SUPER-AD: Semantic Uncertainty-aware Planning for End-to-End Robust Autonomous Driving
- 2026Planning-Oriented End-to-End Autonomous Driving: Architectures, Evaluation, and Emerging Paradigms