mlsanityData to 14 Oct 2026

ILabel: Interactive Neural Scene Labelling

arXiv 2111.14637 ↗First posted 29 Nov 2021cs.CVcited by 28

Not on the map. Why?

Abstract (arXiv)

Joint representation of geometry, colour and semantics using a 3D neural field enables accurate dense labelling from ultra-sparse interactions as a user reconstructs a scene in real-time using a handheld RGB-D sensor. Our iLabel system requires no training data, yet can densely label scenes more accurately than standard methods trained on large, expensively labelled image datasets. Furthermore, it works in an 'open set' manner, with semantic classes defined on the fly by the user. ILabel's underlying model is a multilayer perceptron (MLP) trained from scratch in real-time to learn a joint neural scene representation. The scene model is updated and visualised in real-time, allowing the user to focus interactions to achieve efficient labelling. A room or similar scene can be accurately labelled into 10+ semantic categories with only a few tens of clicks. Quantitative labelling accuracy scales powerfully with the number of clicks, and rapidly surpasses standard pre-trained semantic segmentation methods. We also demonstrate a hierarchical labelling variant.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 28 of its 47 references that are in the map. First posted on arXiv 29 Nov 2021.

Builds on

28 of 47 references.

References · 28

  1. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  2. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  3. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  4. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  5. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  6. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 904
  7. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  8. 2016ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Camerascited by 806
  9. 2015Holistically-Nested Edge Detectioncited by 592
  10. 2020A Survey of Deep Active Learningcited by 554
  11. 2021iMAP: Implicit Mapping and Positioning in Real-Timecited by 476
  12. 2021In-Place Scene Labelling and Understanding with Implicit Scene Representationcited by 415
16 more
  1. 2018CodeSLAM - Learning a Compact, Optimisable Representation for Dense Visual SLAMcited by 152
  2. 2016SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networkscited by 135
  3. 2014Learning Rich Features from RGB-D Images for Object Detection and Segmentationcited by 127
  4. 2020Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentationcited by 122
  5. 2020DeepFactors: Real-Time Probabilistic Dense Monocular SLAMcited by 117
  6. 2016BundleFusion: Real-time Globally Consistent 3D Reconstruction using On-the-fly Surface Re-integrationcited by 116
  7. 2018Fusion++: Volumetric Object-Level SLAMcited by 98
  8. 2019PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Thingscited by 98
  9. 2018In Defense of Classical Image Processing: Fast Depth Completion on the CPUcited by 96
  10. 2020NodeSLAM: Neural Object Descriptors for Multi-View Shape Reconstructioncited by 61
  11. 2019SceneCode: Monocular Dense Semantic Reconstruction using Learned Encoded Scene Representationscited by 37
  12. 2016Meaningful Maps With Object-Oriented Semantic Mappingcited by 31
  13. 2019Incremental Class Discovery for Semantic Segmentation with RGBD Sensingcited by 13
  14. 2015SemanticPaint: A Framework for the Interactive Segmentation of 3D Scenescited by 8
  15. 2016A Robust 3D-2D Interactive Tool for Scene Segmentation and Annotationcited by 3
  16. 2021Hierarchical Pyramid Representations for Semantic Segmentationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 13; 2023: 7; 2024: 5; 2025: 2; 2026: 1.

28 papers.

Sort:

  1. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 371
  2. 2022Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representationscited by 196
  3. 2022CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memorycited by 165
  4. 2022SPIn-NeRF: Multiview Segmentation and Perceptual Inpainting with Neural Radiance Fieldscited by 144
  5. 2022DM-NeRF: 3D Scene Geometry Decomposition and Manipulation from 2D Imagescited by 69
  6. 2023vMAP: Vectorised Object Mapping for Neural Field SLAMcited by 67
  7. 2022LaTeRF: Label and Text Driven Object Radiance Fieldscited by 38
  8. 2022Feature-Realistic Neural Fusion for Real-Time, Open Set Scene Understandingcited by 37
  9. 2022Interactive Object Segmentation in 3D Point Cloudscited by 23
  10. 2023VL-Fields: Towards Language-Grounded Neural Implicit Spatial Representationscited by 23
  11. 2022Unsupervised Continual Semantic Adaptation through Neural Renderingcited by 22
  12. 2022Multi-Object Navigation with dynamically learned neural implicit representationscited by 21
16 more
  1. 2024Neural Fields in Robotics: A Surveycited by 21
  2. 2024NeRFs in Robotics: A Surveycited by 16
  3. 2023LABELMAKER: Automatic Semantic Label Generation from RGB-D Trajectoriescited by 14
  4. 2023AutoNeRF: Training Implicit Scene Representations with Autonomous Agentscited by 10
  5. 2024Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Reviewcited by 9
  6. 2023Neural Radiance Fields: Past, Present, and Futurecited by 8
  7. 2022Baking in the Feature: Accelerating Volumetric Segmentation by Rendering Feature Mapscited by 7
  8. 2025WildSeg3D: Segment Any 3D Objects in the Wild from 2D Imagescited by 6
  9. 2023Reconstructive Latent-Space Neural Radiance Fields for Efficient 3D Scene Representationscited by 5
  10. 2024Semantic Is Enough: Only Semantic Information For NeRF Reconstructioncited by 5
  11. 2024Self-supervised Learning of Neural Implicit Feature Fields for Camera Pose Refinementcited by 5
  12. 2022Real-time Mapping of Physical Scene Properties with an Autonomous Robot Experimentercited by 4
  13. 2023Factored Neural Representation for Scene Understandingcited by 4
  14. 2022You Only Label Once: 3D Box Adaptation from Point Cloud to Image via Semi-Supervised Learningcited by 1
  15. 2025LiLMaps: Learnable Implicit Language Mapscited by 0
  16. 2026SemanticFeels: Semantic Labeling during In-Hand Manipulationcited by 0