mlsanityData to 7 Oct 2026

Pixel2Mesh++: 3D Mesh Generation and Refinement from Multi-View Images

arXiv 2204.09866 ↗First posted 21 Apr 2022cs.CVcited by 21

Not on the map. Why?

Abstract (arXiv)

We study the problem of shape generation in 3D mesh representation from a small number of color images with or without camera poses. While many previous works learn to hallucinate the shape directly from priors, we adopt to further improve the shape quality by leveraging cross-view information with a graph convolution network. Instead of building a direct mapping function from images to 3D shape, our model learns to predict series of deformations to improve a coarse shape iteratively. Inspired by traditional multiple view geometry methods, our network samples nearby area around the initial mesh's vertex locations and reasons an optimal deformation using perceptual feature statistics built from multiple input images. Extensive experiments show that our model produces accurate 3D shapes that are not only visually plausible from the input perspectives, but also well aligned to arbitrary viewpoints. With the help of physically driven architecture, our model also exhibits generalization capability across different semantic categories, and the number of input images. Model analysis experiments show that our model is robust to the quality of the initial mesh and the error of camera pose, and can be combined with a differentiable renderer for test-time optimization.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 60 of its 85 references that are in the map. First posted on arXiv 21 Apr 2022.

Builds on

60 of 85 references.

References · 60

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  3. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  4. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  5. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  6. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  7. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  8. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  9. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  10. 2016Geometric deep learning: going beyond Euclidean datacited by 1,042
  11. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  12. 2020Neural Sparse Voxel Fieldscited by 959
38 more
  1. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  2. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 780
  3. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  4. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  5. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  6. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  7. 2015Multi-view Convolutional Neural Networks for 3D Shape Recognitioncited by 564
  8. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  9. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
  10. 2021Neural Fields in Visual Computing and Beyondcited by 539
  11. 2017Neural 3D Mesh Renderercited by 437
  12. 2019SAL: Sign Agnostic Learning of Shapes from Raw Datacited by 389
  13. 2018PVNet: Pixel-wise Voting Network for 6DoF Pose Estimationcited by 385
  14. 2017End-to-End Learning of Geometry and Context for Deep Stereo Regressioncited by 359
  15. 2016OctNet: Learning Deep 3D Representations at High Resolutionscited by 340
  16. 2018ABC: A Big CAD Model Dataset For Geometric Deep Learningcited by 339
  17. 2019DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstructioncited by 332
  18. 2018AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networkscited by 319
  19. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  20. 2018DeepMVS: Learning Multi-view Stereopsiscited by 277
  21. 2019What Do Single-view 3D Reconstruction Networks Learn?cited by 259
  22. 2019Learning Shape Templates with Structured Implicit Functionscited by 242
  23. 2019Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderercited by 221
  24. 2017Learning a Multi-View Stereo Machinecited by 194
  25. 2017Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputscited by 189
  26. 2016Learning Shape Abstractions by Assembling Volumetric Primitivescited by 181
  27. 2016Learning a Predictable and Generative Vector Representation for Objectscited by 173
  28. 2019CvxNet: Learnable Convex Decompositioncited by 173
  29. 2019Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboidscited by 123
  30. 2019DPSNet: End-to-end Deep Plane Sweep Stereocited by 115
  31. 2019Pixel2Mesh++: Multi-View 3D Mesh Generation via Deformationcited by 103
  32. 2021Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networkscited by 101
  33. 2020Shape and Viewpoint without Keypointscited by 92
  34. 2017Hierarchical Surface Prediction for 3D Object Reconstructioncited by 89
  35. 2018Unsupervised Learning of Shape and Pose with Differentiable Point Cloudscited by 89
  36. 2019Controlling Neural Level Setscited by 76
  37. 2021Unsupervised Learning of 3D Object Categories from Videos in the Wildcited by 67
  38. 2021Shelf-Supervised Mesh Prediction in the Wildcited by 65

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 7; 2024: 8; 2025: 3; 2026: 2.

21 papers.

Sort:

By area:

  1. 2023ShapeGPT: 3D Shape Generation with A Unified Multi-modal Language Modelcited by 38
  2. 2024CharacterGen: Efficient 3D Character Generation from Single Images with Multi-View Pose Canonicalizationcited by 28
  3. 2022Points2NeRF: Generating Neural Radiance Fields from 3D point cloudcited by 11
  4. 2025Not All Frame Features Are Equal: Video-to-4D Generation via Decoupling Dynamic-Static Featurescited by 10
  5. 2023Self-supervised Adaptive Pre-training of Multilingual Speech Models for Language and Dialect Identificationcited by 2
  6. 2026TourPhysics: Bringing Physics to World Models for Exploration and Manipulation from a Single Imagecited by 2
  7. 2023Intelligent Generation of Graphical Game Assets: A Conceptual Framework and Systematic Review of the State of the Artcited by 1
  8. 2024DiffPoint: Single and Multi-view Point Cloud Reconstruction with ViT Based Diffusion Modelcited by 1
  9. 2023Enhancing Multilingual Speech Recognition through Language Prompt Tuning and Frame-Level Language Adaptercited by 0
  10. 2023GMTR: Graph Matching Transformerscited by 0
  11. 2023EvaSurf: Efficient View-Aware Implicit Textured Surface Reconstructioncited by 0
  12. 2023DarkShot: Lighting Dark Images with Low-Compute and High-Qualitycited by 0
9 more
  1. 2024TRLS: A Time Series Representation Learning Framework via Spectrogram for Medical Signal Processingcited by 0
  2. 2024TEN-GUARD: Tensor Decomposition for Backdoor Attack Detection in Deep Neural Networkscited by 0
  3. 2024T-Pixel2Mesh: Combining Global and Local Transformer for 3D Mesh Generation from a Single Imagecited by 0
  4. 2024Strategic Arms with Side Communication Prevail Over Low-Regret MAB Algorithmscited by 0
  5. 2024Towards General Deepfake Detection with Dynamic Curriculumcited by 0
  6. 2024Attr-Int: A Simple and Effective Entity Alignment Framework for Heterogeneous Knowledge Graphscited by 0
  7. 2025Physically Plausible Human-Object Rendering from Sparse Views via 3D Gaussian Splattingcited by 0
  8. 2025Mask6D: Masked Pose Priors For 6D Object Pose Estimationcited by 0
  9. 2026Point Diffusion Mamba: Unified Diffusion-State-Space Modeling for Single-View 3D Reconstruction under Data Scarcitycited by 0