mlsanityData to 14 Oct 2026

DeFormer: Integrating Transformers with Deformable Models for 3D Shape Abstraction from a Single Image

arXiv 2309.12594 ↗First posted 22 Sep 2023cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

Accurate 3D shape abstraction from a single 2D image is a long-standing problem in computer vision and graphics. By leveraging a set of primitives to represent the target shape, recent methods have achieved promising results. However, these methods either use a relatively large number of primitives or lack geometric flexibility due to the limited expressibility of the primitives. In this paper, we propose a novel bi-channel Transformer architecture, integrated with parameterized deformable models, termed DeFormer, to simultaneously estimate the global and local deformations of primitives. In this way, DeFormer can abstract complex object shapes while using a small number of primitives which offer a broader geometry coverage and finer details. Then, we introduce a force-driven dynamic fitting and a cycle-consistent re-projection loss to optimize the primitive parameters. Extensive experiments on ShapeNet across various settings show that DeFormer achieves better reconstruction accuracy over the state-of-the-art, and visualizes with consistent semantic correspondences for improved interpretability.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 51 of its 74 references that are in the map. First posted on arXiv 22 Sep 2023.

Builds on

51 of 74 references.

References · 51

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  5. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  6. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,332
  7. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  8. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,205
  9. 2018Self-Attention with Relative Position Representationscited by 1,120
  10. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  11. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 979
  12. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
38 more
  1. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 651
  2. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 588
  3. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  4. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  5. 2019DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstructioncited by 333
  6. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  7. 2018Analyzing Inverse Problems with Invertible Neural Networkscited by 242
  8. 2019Learning Shape Templates with Structured Implicit Functionscited by 242
  9. 2017Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputscited by 189
  10. 2019Unsupervised Learning of Probabilistic Diffeomorphic Registration for Images and Surfacescited by 186
  11. 2016Learning Shape Abstractions by Assembling Volumetric Primitivescited by 181
  12. 2019CvxNet: Learnable Convex Decompositioncited by 174
  13. 2019Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboidscited by 124
  14. 20163D Shape Induction from 2D Views of Multiple Objectscited by 121
  15. 2020Deformed Implicit Field: Modeling 3D Shapes with Learned Dense Correspondencecited by 119
  16. 2018Unsupervised Learning for Fast Probabilistic Diffeomorphic Registrationcited by 116
  17. 2019Deep Mesh Reconstruction from Single RGB Images via Topology Modification Networkscited by 106
  18. 2021Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networkscited by 103
  19. 2017Hierarchical Surface Prediction for 3D Object Reconstructioncited by 89
  20. 2019BAE-NET: Branched Autoencoder for Shape Co-Segmentationcited by 89
  21. 2020Deep Implicit Templates for 3D Shape Representationcited by 86
  22. 2020DualSDF: Semantic Shape Manipulation using a Two-Level Representationcited by 82
  23. 20173D-PRNN: Generating Shape Primitives with Recurrent Neural Networkscited by 77
  24. 2020Learning Unsupervised Hierarchical Part Decomposition of 3D Objects from a Single RGB Imagecited by 72
  25. 2016Unsupervised Learning of 3D Structure from Imagescited by 67
  26. 2018Im2Struct: Recovering 3D Shape Structure from a Single RGB Imagecited by 55
  27. 2022A Data-scalable Transformer for Medical Image Segmentation: Architecture, Model Efficiency, and Benchmarkcited by 45
  28. 2020D2IM-Net: Learning Detail Disentangled Implicit Fields from Single Imagescited by 42
  29. 2023PartNeRF: Generating Part-Aware Editable 3D Shapes without 3D Supervisioncited by 36
  30. 2021DeepTag: An Unsupervised Deep Learning Method for Motion Tracking on Cardiac Tagging Magnetic Resonance Imagescited by 26
  31. 2023Steering Prototypes with Prompt-tuning for Rehearsal-free Continual Learningcited by 22
  32. 2022TransFusion: Multi-view Divergent Fusion for Medical Image Segmentation with Transformerscited by 21
  33. 2022Biomedical image analysis competitions: The state of current participation practicecited by 20
  34. 2022DeepRecon: Joint 2D Cardiac Segmentation and 3D Volume Reconstruction via A Structure-Specific Generative Methodcited by 17
  35. 2020End-to-End Trainable Deep Active Contour Models for Automated Image Segmentation: Delineating Buildings in Aerial Imagerycited by 16
  36. 2022Region Proposal Rectification Towards Robust Instance Segmentation of Biological Imagescited by 13
  37. 2023MovingParts: Motion-based 3D Part Discovery in Dynamic Radiance Fieldcited by 12
  38. 2023Dealing With Heterogeneous 3D MR Knee Images: A Federated Few-Shot Learning Method With Dual Knowledge Distillationcited by 11

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 4; 2026: 3.

12 papers.

  1. 2023AVID: Any-Length Video Inpainting with Diffusion Modelcited by 82
  2. 2024Layout Agnostic Scene Text Image Synthesis with Diffusion Modelscited by 14
  3. 2024A Review of Transformer-Based Models for Computer Vision Tasks: Capturing Global Context and Spatial Relationshipscited by 11
  4. 2024Self-supervised Learning of Hybrid Part-aware 3D Representations of 2D Gaussians and Superquadricscited by 8
  5. 2025LUCAS: Layered Universal Codec Avatarscited by 7
  6. 2024Instantaneous Perception of Moving Objects in 3Dcited by 3
  7. 2025Snapmoji: Instant Generation of Animatable Dual-Stylized Avatarscited by 2
  8. 2025T2Bs: Text-to-Character Blendshapes via Video Generationcited by 2
  9. 2025Large Sign Language Models: Toward 3D American Sign Language Translationcited by 1
  10. 2026D-Prism: Differentiable Primitives for Structured Dynamic Modelingcited by 0
  11. 2026Geometry-Guided Modeling of Foundation Features Enables Generalizable Object Shape Deformation Learningcited by 0
  12. 2026Organize Primitives into Semantic Parts: Reinforcement Reasoning for 3D Segmentationcited by 0