mlsanityData to 14 Oct 2026

Which One? Leveraging Context Between Objects and Multiple Views for Language Grounding

arXiv 2311.06694 ↗First posted 12 Nov 2023cs.CL cs.AI cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

When connecting objects and their language referents in an embodied 3D environment, it is important to note that: (1) an object can be better characterized by leveraging comparative information between itself and other objects, and (2) an object's appearance can vary with camera position. As such, we present the Multi-view Approach to Grounding in Context (MAGiC), which selects an object referent based on language that distinguishes between two similar objects. By pragmatically reasoning over both objects and across multiple views of those objects, MAGiC improves over the state-of-the-art model on the SNARE object reference task with a relative error reduction of 12.9% (representing an absolute improvement of 2.7%). Ablation studies show that reasoning jointly over object referent candidates and multiple views of each object both contribute to improved accuracy. Code: https://github.com/rcorona/magicsnare/

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 40 of its 56 references that are in the map. First posted on arXiv 12 Nov 2023.

Builds on

40 of 56 references.

References · 40

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  5. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  6. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  7. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  8. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  9. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,362
  10. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  11. 2016Modeling Context in Referring Expressionscited by 1,063
  12. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
28 more
  1. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 699
  2. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 658
  3. 2019Deep Learning for 3D Point Clouds: A Surveycited by 558
  4. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 551
  5. 2022Multimodal Learning with Transformers: A Surveycited by 400
  6. 2020VirTex: Learning Visual Representations from Textual Annotationscited by 339
  7. 2020MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysiscited by 265
  8. 2021Mesh Graphormercited by 265
  9. 2021Anticipative Video Transformercited by 200
  10. 2022Multi-View Transformer for 3D Visual Groundingcited by 140
  11. 2021Evaluating CLIP: Towards Characterization of Broader Capabilities and Downstream Implicationscited by 128
  12. 2020ACRONYM: A Large-Scale Grasp Dataset Based on Simulationcited by 117
  13. 2016GuessWhat?! Visual object discovery through multi-modal dialoguecited by 113
  14. 2019Neural 3D Morphable Models: Spiral Convolutional Networks for 3D Shape Representation Learning and Generationcited by 90
  15. 2022Cross-Attention of Disentangled Modalities for 3D Human Mesh Recovery with Transformerscited by 81
  16. 2019Zero-Shot Grounding of Objects from Natural Language Queriescited by 79
  17. 2016Reasoning About Pragmatics with Neural Listeners and Speakerscited by 68
  18. 2017Colors in Context: A Pragmatic Neural Model for Grounded Language Understandingcited by 57
  19. 2019ShapeGlot: Learning Language for Shape Differentiationcited by 54
  20. 2021Worst of Both Worlds: Biases Compound in Pre-trained Vision-and-Language Modelscited by 52
  21. 2018Cooperative Holistic Scene Understanding: Unifying 3D Object, Layout, and Camera Pose Estimationcited by 51
  22. 2020Improving Weakly Supervised Visual Grounding by Contrastive Knowledge Distillationcited by 51
  23. 2021Language Grounding with 3D Objectscited by 43
  24. 2021PartGlot: Learning Shape Part Segmentation from Language Reference Gamescited by 30
  25. 2021LegoFormer: Transformers for Block-by-Block Multi-view 3D Reconstructioncited by 25
  26. 2022FairCLIP: Social Bias Elimination based on Attribute Prototype Learning and Representation Neutralizationcited by 20
  27. 2022Learning to Mediate Disparities Towards Pragmatic Communicationcited by 14
  28. 2016DeepContext: Context-Encoding Neural Pathways for 3D Holistic Scene Understandingcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2026Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisonscited by 60
  2. 2025Vision-and-Language Navigation with Analogical Textual Descriptions in LLMscited by 3
  3. 2025Why Are You Wrong? Counterfactual Explanations for Language Grounding with 3D Objectscited by 1