mlsanityData to 14 Oct 2026

IKEA-Manual: Seeing Shape Assembly Step by Step

arXiv 2302.01881 ↗First posted 3 Feb 2023cs.CV cs.AIcited by 22

Not on the map. Why?

Abstract (arXiv)

Human-designed visual manuals are crucial components in shape assembly activities. They provide step-by-step guidance on how we should move and connect different parts in a convenient and physically-realizable way. While there has been an ongoing effort in building agents that perform assembly tasks, the information in human-design manuals has been largely overlooked. We identify that this is due to 1) a lack of realistic 3D assembly objects that have paired manuals and 2) the difficulty of extracting structured information from purely image-based manuals. Motivated by this observation, we present IKEA-Manual, a dataset consisting of 102 IKEA objects paired with assembly manuals. We provide fine-grained annotations on the IKEA objects and assembly manuals, including decomposed assembly parts, assembly plans, manual segmentation, and 2D-3D correspondence between 3D parts and visual manuals. We illustrate the broad application of our dataset on four tasks related to shape assembly: assembly plan generation, part segmentation, pose estimation, and 3D part assembly.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 26 of its 55 references that are in the map. First posted on arXiv 3 Feb 2023.

Builds on

26 of 55 references.

References · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  4. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  5. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  6. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,223
  7. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  8. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 526
  9. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  10. 2018PVNet: Pixel-wise Voting Network for 6DoF Pose Estimationcited by 385
  11. 2018ABC: A Big CAD Model Dataset For Geometric Deep Learningcited by 341
  12. 2017BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depthcited by 225
14 more
  1. 2018Pix3D: Dataset and Methods for Single-Image 3D Shape Modelingcited by 218
  2. 2016Thingi10K: A Dataset of 10,000 3D-Printing Modelscited by 183
  3. 2017Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistencycited by 177
  4. 2021JoinABLe: Learning Bottom-up Assembly of Parametric CAD Jointscited by 78
  5. 2019IKEA Furniture Assembly Environment for Long-Horizon Complex Manipulation Taskscited by 69
  6. 2020Generative 3D Part Assembly via Dynamic Graph Learningcited by 67
  7. 2022Factory: Fast Contact for Robotic Assemblycited by 54
  8. 2020Learning 3D Part Assembly from a Single Imagecited by 52
  9. 2019Structured agents for physical constructioncited by 50
  10. 2019Pose from Shape: Deep Pose Estimation for Arbitrary 3D Objectscited by 39
  11. 2021RGL-NET: A Recurrent Graph Learning framework for Progressive Part Assemblycited by 36
  12. 2022Blocks Assemble! Learning to Assemble with Large-Scale Structured Reinforcement Learningcited by 22
  13. 2021Brick-by-Brick: Combinatorial Construction with Deep Reinforcement Learningcited by 19
  14. 2021IKEA Object State Dataset: A 6DoF object pose estimation dataset and benchmark for multi-state assembly objectscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 9; 2025: 7; 2026: 5.

22 papers.

Sort:

By area:

  1. 2023ZeroPS: High-quality Cross-modal Knowledge Transfer for Zero-Shot 3D Part Segmentationcited by 18
  2. 20243D Part Segmentation via Geometric Aggregation of 2D Visual Featurescited by 15
  3. 2025Manual2Skill: Learning to Read Manuals and Acquire Robotic Skills for Furniture Assembly Using Vision-Language Modelscited by 14
  4. 2025Rectified Point Flow: Generic Point Cloud Pose Estimationcited by 14
  5. 2025Two by Two: Learning Multi-Task Pairwise Objects Assembly for Generalizable Robot Manipulationcited by 13
  6. 2024SPAFormer: Sequential 3D Part Assembly with Transformerscited by 12
  7. 2024Part2Object: Hierarchical Unsupervised 3D Instance Segmentationcited by 12
  8. 2024IKEA Manuals at Work: 4D Grounding of Assembly Instructions on Internet Videoscited by 12
  9. 20243x2: 3D Object Part Segmentation by 2D Semantic Correspondencescited by 10
  10. 2024GBOT: Graph-Based 3D Object Tracking for Augmented Reality-Assisted Assembly Guidancecited by 7
  11. 2024Manual-PA: Learning 3D Part Assembly from Instruction Diagramscited by 7
  12. 2024ASDF: Assembly State Detection Utilizing Late Fusion by Integrating 6D Pose Estimationcited by 6
10 more
  1. 2025BiAssemble: Learning Collaborative Affordance for Bimanual Geometric Assemblycited by 5
  2. 2025Manual2Skill++: Connector-Aware General Robotic Assembly from Instruction Manuals via Vision-Language Modelscited by 5
  3. 2024SCANet: Correcting LEGO Assembly Errors with Self-Correct Assembly Networkcited by 3
  4. 2026Retrieval-Augmented Robots via Retrieve-Reason-Actcited by 2
  5. 2026AssemblyBench: Physics-Aware Assembly of Complex Industrial Objectscited by 2
  6. 2026OmniCAD: A Large-Scale Benchmark for 3D Spatial Reasoning in Robotics Assembliescited by 1
  7. 2025Query-Centric Diffusion Policy for Generalizable Robotic Assemblycited by 0
  8. 2025PinPoint3D: Fine-Grained 3D Part Segmentation from a Few Clickscited by 0
  9. 2026Benchmarking and Mechanistic Analysis of Vision-Language Models for Cross-Depiction Assembly Instruction Alignmentcited by 0
  10. 2026AssemblyWorld: Rethinking 3D Assembly with General-Purpose Agentscited by 0