mlsanityData to 7 Oct 2026

Cross-view Transformers for real-time Map-view Semantic Segmentation

arXiv 2205.02833 ↗First posted 5 May 2022cs.CV cs.AIcited by 264

Not on the map. Why?

Cited by 22 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present cross-view transformers, an efficient attention-based model for map-view semantic segmentation from multiple cameras. Our architecture implicitly learns a mapping from individual camera views into a canonical map-view representation using a camera-aware cross-view attention mechanism. Each camera uses positional embeddings that depend on its intrinsic and extrinsic calibration. These embeddings allow a transformer to learn the mapping across different views without ever explicitly modeling it geometrically. The architecture consists of a convolutional image encoder for each view and cross-view transformer layers to infer a map-view semantic segmentation. Our model is simple, easily parallelizable, and runs in real-time. The presented architecture performs at state-of-the-art on the nuScenes dataset, with 4x faster inference speeds. Code is available at https://github.com/bradyz/cross_view_transformers.

Key facts. Cited by 264 papers in the map (data through 7 Oct 2026). Builds on 32 of its 49 references that are in the map. First posted on arXiv 5 May 2022.

Builds on

32 of 49 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  3. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  5. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  6. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  7. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  8. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  9. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  10. 2019Objects as Pointscited by 1,172
  11. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  12. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 819
20 more
  1. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
  2. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 728
  3. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 721
  4. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 455
  5. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  6. 2020One Thousand and One Hours: Self-driving Motion Prediction Datasetcited by 239
  7. 2020Atlas: End-to-End 3D Scene Reconstruction from Posed Imagescited by 206
  8. 2018Orthographic Feature Transform for Monocular 3D Object Detectioncited by 201
  9. 2017Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Ratescited by 196
  10. 2020Predicting Semantic Map Representations from Images using Pyramid Occupancy Networkscited by 175
  11. 2019Cross-view Semantic Segmentation for Sensing Surroundingscited by 173
  12. 2019Accurate Monocular Object Detection via Color-Embedded 3D Reconstruction for Autonomous Drivingcited by 137
  13. 2018ROI-10D: Monocular Lifting of 2D Detection to 6D Pose and Metric Shapecited by 123
  14. 2019Monocular 3D Object Detection with Pseudo-LiDAR Point Cloudcited by 113
  15. 20183D-LaneNet: End-to-End 3D Multiple Lane Detectioncited by 106
  16. 2018Monocular Semantic Occupancy Grid Mapping with Convolutional Variational Encoder-Decoder Networkscited by 84
  17. 2020MonoLayout: Amodal scene layout from a single imagecited by 49
  18. 2019A Geometric Approach to Obtain a Bird's Eye View from an Imagecited by 34
  19. 2020Driving among Flatmobiles: Bird-Eye-View occupancy grids from a monocular camera for holistic trajectory planningcited by 24
  20. 2021Monocular 3D Vehicle Detection Using Uncalibrated Traffic Cameras through Homographycited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 27; 2023: 82; 2024: 91; 2025: 41; 2026: 23.

264 papers.

Sort:

By area:

  1. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  2. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 613
  3. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  4. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 351
  5. 2022MapTR: Structured Modeling and Learning for Online Vectorized HD Map Constructioncited by 311
  6. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 309
  7. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  8. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 287
  9. 2023MagicDrive: Street View Generation with Diverse 3D Geometry Controlcited by 250
  10. 2022VectorMapNet: End-to-end Vectorized HD Map Learningcited by 243
  11. 2022CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformerscited by 227
  12. 2023Scene as Occupancycited by 208
38 more
  1. 2023MapTRv2: An End-to-End Framework for Online Vectorized HD Map Constructioncited by 192
  2. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 168
  3. 2023Panacea: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 165
  4. 2023V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perceptioncited by 155
  5. 2023DDP: Diffusion Model for Dense Visual Predictioncited by 148
  6. 2022Simple-BEV: What Really Matters for Multi-Sensor BEV Perception?cited by 128
  7. 2022Vision-Centric BEV Perception: A Surveycited by 126
  8. 2022Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipecited by 122
  9. 2023Consistent View Synthesis with Pose-Guided Diffusion Modelscited by 117
  10. 2023PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentationcited by 114
  11. 2022PlanT: Explainable Planning Transformers via Object-Level Representationscited by 105
  12. 2023Street-View Image Generation from a Bird's-Eye View Layoutcited by 102
  13. 2023BEVControl: Accurately Controlling Street-view Elements with Multi-perspective Consistency via BEV Sketch Layoutcited by 102
  14. 2024UniScene: Unified Occupancy-centric Driving Scene Generationcited by 102
  15. 2024SparseOcc: Rethinking Sparse Latent Representation for Vision-Based Semantic Occupancy Predictioncited by 94
  16. 2024MagicDrive-V2: High-Resolution Long Video Generation for Autonomous Driving with Adaptive Controlcited by 90
  17. 2023Symphonize 3D Semantic Scene Completion with Contextual Instance Queriescited by 88
  18. 2023Collaborative Perception in Autonomous Driving: Methods, Datasets and Challengescited by 87
  19. 2023Collaboration Helps Camera Overtake LiDAR in 3D Detectioncited by 81
  20. 2023CRN: Camera Radar Net for Accurate, Robust, Efficient 3D Perceptioncited by 81
  21. 2022Efficient and Robust 2D-to-BEV Representation Learning via Geometry-guided Kernel Transformercited by 80
  22. 2023End-to-End Vectorized HD-map Construction with Piecewise Bezier Curvecited by 80
  23. 2023DrivingDiffusion: Layout-Guided multi-view driving scene video generation with latent diffusion modelcited by 75
  24. 2023GTA: A Geometry-Aware Attention Mechanism for Multi-View Transformerscited by 75
  25. 2023UniTR: A Unified and Efficient Multi-Modal Transformer for Bird's-Eye-View Representationcited by 74
  26. 2024MagicDrive3D: Controllable 3D Generation for Any-View Rendering in Street Scenescited by 70
  27. 2023Neural Map Prior for Autonomous Drivingcited by 68
  28. 20253D and 4D World Modeling: A Surveycited by 64
  29. 2023LaneSegNet: Map Learning with Lane Segment Perception for Autonomous Drivingcited by 57
  30. 2022Uncertainty-aware Vision-based Metric Cross-view Geolocalizationcited by 54
  31. 2023Online Map Vectorization for Autonomous Driving: A Rasterization Perspectivecited by 54
  32. 2023Graph-based Topology Reasoning for Driving Scenescited by 53
  33. 2024POP-3D: Open-Vocabulary 3D Occupancy Prediction from Imagescited by 52
  34. 2024FastOcc: Accelerating 3D Occupancy Prediction by Fusing the 2D Bird's-Eye View and Perspective Viewcited by 51
  35. 2023Lane Graph as Path: Continuity-preserving Path-wise Modeling for Online Lane Graph Constructioncited by 50
  36. 2024HIMap: HybrId Representation Learning for End-to-end Vectorized HD Map Constructioncited by 48
  37. 2024Leveraging Enhanced Queries of Point Sets for Vectorized Map Constructioncited by 46
  38. 2023Boosting 3-DoF Ground-to-Satellite Camera Localization Accuracy via Geometry-Guided Cross-View Transformercited by 45

50 most cited of 264 All 264 (JSON)