mlsanityData to 14 Oct 2026

Disc3D: Automatic Curation of High-Quality 3D Dialog Data via Discriminative Object Referring

arXiv 2511.18817 ↗First posted 24 Nov 2025cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

3D Multi-modal Large Language Models (MLLMs) still lag behind their 2D peers, largely because large-scale, high-quality 3D scene-dialogue datasets remain scarce. Prior efforts hinge on expensive human annotation and leave two key ambiguities unresolved: viewpoint ambiguity, where spatial language presumes unknown camera poses, and object referring ambiguity, where non-exclusive descriptions blur the line between targets and distractors. We therefore present a fully automated pipeline that converts raw 3D scans into unambiguous, high-quality dialogue data at a fraction of the previous cost. By synergizing rule-based constraints with 2D MLLMs and LLMs, the pipeline enables controllable, scalable generation without human intervention. The pipeline comprises four stages: (1) meta-annotation collection harvesting object-, frame-, and scene-level captions, (2) scene graph construction with relation correction to capture proximal object relations, (3) discriminative object referring that generates exclusive and compact descriptions, and (4) multi-task data generation synthesizing diverse dialogues. Our pipeline systematically mitigates inherent flaws in source datasets and produces the final Disc3D dataset, over 2 million samples in 25K hybrid 3D scenes, spanning scene, view, and object captioning, visual grounding, and five object-centric QA tasks. Extensive experiments demonstrate that training with Disc3D yields consistent, significant improvements on both public benchmarks and our multifaceted Disc3D-QA tasks. Code, data, and models will be publicly available.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 40 of its 47 references that are in the map. First posted on arXiv 24 Nov 2025.

Builds on

40 of 47 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Ross3D: Reconstructive Visual Instruction Tuning with 3D-Awareness2 Apr
  2. 2025Inst3D-LMM: Instance-Aware 3D Scene Understanding with Multi-modal Instruction Tuning1 Mar
  3. 2025GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Models2 Jan
  4. 20253D-LLaVA: Towards Generalist 3D LMMs with Omni Superpoint Transformer2 Jan
  5. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec
  6. 2024Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understanding30 Nov
  7. 2024LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness26 Sep
  8. 2023GPT4Point: A Unified Framework for Point-Language Understanding and Generation5 Dec
  9. 2023LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning30 Nov

Other references · 31

  1. 2025Qwen2.5-VL Technical Reportcited by 5,587
  2. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  3. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  4. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  5. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
  6. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
  7. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
  8. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 454
  9. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 452
  10. 2025Kimi-VL Technical Reportcited by 388
  11. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 380
  12. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 346
19 more
  1. 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 333
  2. 2022SQA3D: Situated Question Answering in 3D Scenescited by 314
  3. 2022Mask3D: Mask Transformer for 3D Semantic Instance Segmentationcited by 303
  4. 2019Structured3D: A Large Photo-realistic Dataset for Structured 3D Modelingcited by 286
  5. 20233D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignmentcited by 204
  6. 2019RIO: 3D Object Instance Re-Localization in Changing Indoor Environmentscited by 188
  7. 2023Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifierscited by 175
  8. 2023Multi3DRefer: Grounding Text Description to Multiple 3D Objectscited by 146
  9. 2021InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referringcited by 141
  10. 2024SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understandingcited by 133
  11. 2021SAT: 2D Semantics Assisted Training for 3D Visual Groundingcited by 124
  12. 20223D-SPS: Single-Stage 3D Visual Grounding via Referred Point Progressive Selectioncited by 110
  13. 2024SceneScript: Reconstructing Scenes With An Autoregressive Structured Language Modelcited by 96
  14. 2022UniT3D: A Unified Transformer for 3D Dense Captioning and Visual Groundingcited by 69
  15. 2024VLM-Grounder: A VLM Agent for Zero-Shot 3D Visual Groundingcited by 56
  16. 2024MMScan: A Multi-Modal 3D Scene Dataset with Hierarchical Grounded Language Annotationscited by 40
  17. 2024Cubify Anything: Scaling Indoor 3D Object Detectioncited by 39
  18. 2024Open-YOLO 3D: Towards Fast and Accurate Open-Vocabulary 3D Instance Segmentationcited by 30
  19. 2025Unveiling the Mist over 3D Vision-Language Understanding: Object-centric Evaluation with Chain-of-Analysiscited by 23

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenes
  2. 2024Grounded 3D-LLM with Referent Tokens
  3. 2026PAR3D: A Unified 3D-MLLM with Part-Aware Representation for Scene Understanding
  4. 2026Ground3D-LMM: Fine-Grained 3D Point Grounding and Spatial Reasoning with LMM
  5. 2025Text-Scene: A Scene-to-Language Parsing Framework for 3D Scene Understanding
  6. 2026Chat-Scene++: Exploiting Context-Rich Object Identification for 3D LLM
  7. 2026MM-Conv: A Multimodal Dataset and Benchmark for Context-Aware Grounding in 3D Dialogue
  8. 2025DenseScan: Advancing 3D Scene Understanding with 2D Dense Annotation