mlsanityData to 7 Oct 2026

Voxel-informed Language Grounding

arXiv 2205.09710 ↗First posted 19 May 2022cs.CL cs.AI cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

Natural language applied to natural 2D images describes a fundamentally 3D world. We present the Voxel-informed Language Grounder (VLG), a language grounding model that leverages 3D geometric information in the form of voxel maps derived from the visual input using a volumetric reconstruction model. We show that VLG significantly improves grounding accuracy on SNARE, an object reference game task. At the time of writing, VLG holds the top place on the SNARE leaderboard, achieving SOTA results with a 2.0% absolute improvement.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 24 of its 36 references that are in the map. First posted on arXiv 19 May 2022.

Builds on

24 of 36 references.

References · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  5. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  6. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  7. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  8. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 446
  9. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
  10. 201912-in-1: Multi-Task Vision and Language Representation Learningcited by 278
  11. 2019Language2Pose: Natural Language Grounded Pose Forecastingcited by 257
  12. 2020Experience Grounds Languagecited by 241
12 more
  1. 2018Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddingscited by 169
  2. 2020Task-Oriented Dialogue as Dataflow Synthesiscited by 122
  3. 2020ACRONYM: A Large-Scale Grasp Dataset Based on Simulationcited by 115
  4. 2019Learning by Abstraction: The Neural State Machinecited by 113
  5. 2021A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Executioncited by 91
  6. 2021LanguageRefer: Spatial-Language Model for 3D Visual Groundingcited by 88
  7. 2019ShapeGlot: Learning Language for Shape Differentiationcited by 54
  8. 2021Language Grounding with 3D Objectscited by 43
  9. 2021PartGlot: Learning Shape Part Segmentation from Language Reference Gamescited by 30
  10. 2021LegoFormer: Transformers for Block-by-Block Multi-view 3D Reconstructioncited by 25
  11. 2019Embodied Language Grounding with 3D Visual Feature Representationscited by 20
  12. 2020Language-Mediated, Object-Centric Representation Learningcited by 11

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 4; 2024: 4; 2025: 1.

11 papers.

  1. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
  2. 2023LERF: Language Embedded Radiance Fieldscited by 538
  3. 2022PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Modelscited by 117
  4. 2024FMGS: Foundation Model Embedded 3D Gaussian Splatting for Holistic 3D Scene Understandingcited by 86
  5. 2023Scene-Driven Multimodal Knowledge Graph Construction for Embodied AIcited by 17
  6. 2023Iterative Superquadric Recomposition of 3D Objects from Multiple Viewscited by 13
  7. 20244-LEGS: 4D Language Embedded Gaussian Splattingcited by 12
  8. 2024TIGER: Text-Instructed 3D Gaussian Retrieval and Coherent Editingcited by 10
  9. 2023DetermiNet: A Large-Scale Diagnostic Dataset for Complex Visually-Grounded Referencing using Determinerscited by 3
  10. 2024Multi-Task Domain Adaptation for Language Grounding with 3D Objectscited by 2
  11. 2025Why Are You Wrong? Counterfactual Explanations for Language Grounding with 3D Objectscited by 1