mlsanityData to 7 Oct 2026

PlaneFormers: From Sparse View Planes to 3D Reconstruction

arXiv 2208.04307 ↗First posted 8 Aug 2022cs.CVcited by 26

Not on the map. Why?

Abstract (arXiv)

We present an approach for the planar surface reconstruction of a scene from images with limited overlap. This reconstruction task is challenging since it requires jointly reasoning about single image 3D reconstruction, correspondence between images, and the relative camera pose between images. Past work has proposed optimization-based approaches. We introduce a simpler approach, the PlaneFormer, that uses a transformer applied to 3D-aware plane tokens to perform 3D reasoning. Our experiments show that our approach is substantially more effective than prior work, and that several 3D-specific design decisions are crucial for its success.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 49 of its 68 references that are in the map. First posted on arXiv 8 Aug 2022.

Builds on

49 of 68 references.

References · 49

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Mask R-CNNcited by 7,259
  3. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  4. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  5. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  6. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  7. 2020Point Transformercited by 1,462
  8. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
  9. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  10. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 983
  11. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 974
  12. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
37 more
  1. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 735
  2. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 731
  3. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  4. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 673
  5. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  6. 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 641
  7. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  8. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  9. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  10. 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
  11. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 417
  12. 2020Local Implicit Grid Representations for 3D Scenescited by 372
  13. 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
  14. 2018DeepMVS: Learning Multi-view Stereopsiscited by 277
  15. 2021NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Videocited by 241
  16. 2020Deep Global Registrationcited by 228
  17. 2020Image Matching across Wide Baselines: From Paper to Practicecited by 224
  18. 2016DeMoN: Depth and Motion Network for Learning Monocular Stereocited by 200
  19. 2021Pixel-Perfect Structure-from-Motion with Featuremetric Refinementcited by 199
  20. 2017Learning a Multi-View Stereo Machinecited by 194
  21. 2020Robust Consistent Video Depth Estimationcited by 188
  22. 2017Learning to Find Good Correspondencescited by 176
  23. 2019Learning Two-View Correspondences and Geometry Using Order-Aware Networkcited by 148
  24. 2020TartanVO: A Generalizable Learning-based VOcited by 107
  25. 2021TransformerFusion: Monocular RGB Scene Reconstruction using Transformerscited by 107
  26. 2018PlaneRCNN: 3D Plane Detection and Reconstruction from a Single Imagecited by 105
  27. 2018PlaneNet: Piece-wise Planar Reconstruction from a Single RGB Imagecited by 77
  28. 2020OASIS: A Large-Scale Dataset for Single Image 3D in the Wildcited by 62
  29. 2014Designing Deep Networks for Surface Normal Estimationcited by 59
  30. 2019Single-Image Piece-wise Planar 3D Reconstruction via Associative Embeddingcited by 56
  31. 2021Consistent Depth of Moving Objects in Videocited by 56
  32. 2021Wide-Baseline Relative Camera Pose Estimation with Directional Learningcited by 49
  33. 2021UnsupervisedR&R: Unsupervised Point Cloud Registration via Differentiable Renderingcited by 44
  34. 2021Extreme Rotation Estimation using Dense Correlation Volumescited by 42
  35. 2021Planar Surface Reconstruction from Sparse Viewscited by 40
  36. 2020Associative3D: Volumetric Reconstruction from Sparse Viewscited by 18
  37. 2020MessyTable: Instance Association in Multiple Camera Viewscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 6; 2024: 11; 2025: 5; 2026: 1.

26 papers.

Sort:

By area:

  1. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  2. 2022The 8-Point Algorithm as an Inductive Bias for Relative Pose Prediction by ViTscited by 38
  3. 2022Last-Mile Embodied Visual Navigationcited by 25
  4. 2022NOPE-SAC: Neural One-Plane RANSAC for Sparse-View Planar 3D Reconstructioncited by 21
  5. 2024AirPlanes: Accurate Plane Estimation via 3D-Consistent Embeddingscited by 13
  6. 2024PlanarSplatting: Accurate Planar Surface Reconstruction in 3 Minutescited by 13
  7. 2023Structural Multiplane Image: Bridging Neural View Synthesis and 3D Reconstructioncited by 12
  8. 2024FAR: Flexible, Accurate and Robust 6DoF Relative Camera Pose Estimationcited by 12
  9. 2025G4Splat: Geometry-Guided Gaussian Splatting with Generative Priorcited by 10
  10. 2023PlaneRecTR++: Unified Query Learning for Joint 3D Planar Reconstruction and Pose Estimationcited by 6
  11. 20243DFIRES: Few Image 3D REconstruction for Scenes with Hidden Surfacecited by 6
  12. 2024Extreme Rotation Estimation in the Wildcited by 6
14 more
  1. 2024AlphaTablets: A Generic Plane Representation for 3D Planar Reconstruction from Monocular Videoscited by 6
  2. 2024Planar Gaussian Splattingcited by 6
  3. 2025Towards In-the-wild 3D Plane Reconstruction from a Single Imagecited by 6
  4. 2023PlanarNeRF: Online Learning of Planar Primitives with Neural Radiance Fieldscited by 5
  5. 2024SRPose: Two-view Relative Pose Estimation with Sparse Keypointscited by 5
  6. 2025PLANA3R: Zero-shot Metric Planar 3D Reconstruction via Feed-Forward Planar Splattingcited by 5
  7. 2024MonoPlane: Exploiting Monocular Geometric Cues for Generalizable 3D Plane Reconstructioncited by 2
  8. 2025Co-VisiON: Co-Visibility ReasONing on Sparse Image Sets of Indoor Scenescited by 2
  9. 2025PixCuboid: Room Layout Estimation from Multi-view Featuremetric Alignmentcited by 2
  10. 20233D detection of roof sections from a single satellite image and application to LOD2-building reconstructioncited by 1
  11. 2024Neural Mesh Fusion: Unsupervised 3D Planar Surface Understandingcited by 1
  12. 2024Indoor 3D Reconstruction with an Unknown Camera-Projector Paircited by 1
  13. 2026Interacted Planes Reveal 3D Line Mappingcited by 1
  14. 2023LoCUS: Learning Multiscale 3D-consistent Features from Posed Imagescited by 0