mlsanityData to 7 Oct 2026

TO-Scene: A Large-scale Dataset for Understanding 3D Tabletop Scenes

arXiv 2203.09440 ↗First posted 17 Mar 2022cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

Many basic indoor activities such as eating or writing are always conducted upon different tabletops (e.g., coffee tables, writing desks). It is indispensable to understanding tabletop scenes in 3D indoor scene parsing applications. Unfortunately, it is hard to meet this demand by directly deploying data-driven algorithms, since 3D tabletop scenes are rarely available in current datasets. To remedy this defect, we introduce TO-Scene, a large-scale dataset focusing on tabletop scenes, which contains 20,740 scenes with three variants. To acquire the data, we design an efficient and scalable framework, where a crowdsourcing UI is developed to transfer CAD objects from ModelNet and ShapeNet onto tables from ScanNet, then the output tabletop scenes are simulated into real scans and annotated automatically. Further, a tabletop-aware learning strategy is proposed for better perceiving the small-sized tabletop instances. Notably, we also provide a real scanned test set TO-Real to verify the practical value of TO-Scene. Experiments show that the algorithms trained on TO-Scene indeed work on the realistic test data, and our proposed tabletop-aware learning strategy greatly improves the state-of-the-art results on both 3D semantic segmentation and object detection tasks. Dataset and code are available at https://github.com/GAP-LAB-CUHK-SZ/TO-Scene.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 40 of its 63 references that are in the map. First posted on arXiv 17 Mar 2022.

Builds on

40 of 63 references.

References · 40

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  3. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  4. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  5. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  6. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
  7. 2020Point Transformercited by 1,462
  8. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  9. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,202
  10. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  11. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 746
  12. 2018PointConv: Deep Convolutional Networks on 3D Point Cloudscited by 656
28 more
  1. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  2. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
  3. 2019Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Datacited by 532
  4. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  5. 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 375
  6. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 360
  7. 2018ABC: A Big CAD Model Dataset For Geometric Deep Learningcited by 339
  8. 20163DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructionscited by 333
  9. 2021An End-to-End Transformer Model for 3D Object Detectioncited by 329
  10. 2021Group-Free 3D Object Detection via Transformerscited by 215
  11. 2021PAConv: Position Adaptive Convolution with Dynamic Kernel Assembling on Point Cloudscited by 211
  12. 2020Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotationscited by 155
  13. 2018Scan2CAD: Learning CAD Model Alignment in RGB-D Scanscited by 142
  14. 2020H3DNet: 3D Object Detection Using Hybrid Geometric Primitivescited by 123
  15. 2013Deep Learning for Detecting Robotic Graspscited by 114
  16. 2021FCAF3D: Fully Convolutional Anchor-Free 3D Object Detectioncited by 97
  17. 2021Bidirectional Projection Network for Cross Dimension Scene Understandingcited by 95
  18. 2020A Closer Look at Local Aggregation Operators in Point Cloud Analysiscited by 87
  19. 2018Jacquard: A Large Scale Dataset for Robotic Grasp Detectioncited by 86
  20. 2019BlenderProccited by 76
  21. 2020OpenRooms: An End-to-End Open Framework for Photorealistic Indoor Scene Datasetscited by 73
  22. 2020Learning Geometry-Disentangled Representation for Complementary Understanding of 3D Object Point Cloudcited by 71
  23. 2020Generative Sparse Detection Networks for 3D Single-shot Object Detectioncited by 69
  24. 2021Back-tracing Representative Points for Voting-based 3D Object Detection in Point Cloudscited by 67
  25. 2016A Large Dataset of Object Scanscited by 64
  26. 2020FPConv: Learning Local Flattening for Point Convolutioncited by 55
  27. 2016Multiview RGB-D Dataset for Object Instance Detectioncited by 26
  28. 2019The RobotriX: An eXtremely Photorealistic and Very-Large-Scale Indoor Dataset of Sequences with Robot Trajectories and Interactionscited by 14

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 5; 2024: 1; 2025: 3.

10 papers.

  1. 2023MVImgNet: A Large-scale Dataset of Multi-view Imagescited by 219
  2. 2025DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learningcited by 62
  3. 2023Synthetic Datasets for Autonomous Driving: A Surveycited by 48
  4. 2023SAMPro3D: Locating SAM Prompts in 3D for Zero-Shot Instance Segmentationcited by 44
  5. 2023EFEM: Equivariant Neural Field Expectation Maximization for 3D Object Segmentation Without Scene Supervisioncited by 18
  6. 2022MM-3DScene: 3D Scene Understanding by Customizing Masked Modeling with Informative-Preserved Reconstruction and Self-Distilled Consistencycited by 12
  7. 20233D Small Object Detection with Dynamic Spatial Pruningcited by 4
  8. 2025Stable-Sim2Real: Exploring Simulation of Real-Captured 3D Data with Two-Stage Depth Diffusioncited by 2
  9. 2025GOPLA: Generalizable Object Placement Learning via Synthetic Augmentation of Human Arrangementcited by 1
  10. 20243D Adaptive Structural Convolution Network for Domain-Invariant Point Cloud Recognitioncited by 0