mlsanityData to 7 Oct 2026

MAttNet: Modular Attention Network for Referring Expression Comprehension

arXiv 1801.08186 ↗First posted 24 Jan 2018cs.CV cs.AI cs.CLcited by 427

Not on the map. Why?

Cited by 54 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this paper, we address referring expression comprehension: localizing an image region described by a natural language expression. While most recent work treats expressions as a single unit, we propose to decompose them into three modular components related to subject appearance, location, and relationship to other objects. This allows us to flexibly adapt to expressions containing different types of information in an end-to-end framework. In our model, which we call the Modular Attention Network (MAttNet), two types of attention are utilized: language-based attention that learns the module weights as well as the word/phrase attention that each module should focus on; and visual attention that allows the subject and relationship modules to focus on relevant image components. Module weights combine scores from all three modules dynamically to output an overall score. Experiments show that MAttNet outperforms previous state-of-art methods by a large margin on both bounding-box-level and pixel-level comprehension tasks. Demo and code are provided.

Key facts. Cited by 427 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Jan 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 66; 2022: 92; 2023: 106; 2024: 69; 2025: 54; 2026: 40.

427 papers.

Sort:

By area:

  1. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  2. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  3. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 811
  4. 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 498
  5. 2021Scaling Open-Vocabulary Image Segmentation with Image-Level Labelscited by 471
  6. 2021PointCLIP: Point Cloud Understanding by CLIPcited by 461
  7. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modelingcited by 429
  8. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 411
  9. 2022Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classificationcited by 355
  10. 2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
  11. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 347
  12. 2021TransVG: End-to-End Visual Grounding with Transformerscited by 269
38 more
  1. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  2. 2023Unleashing Text-to-Image Diffusion Models for Visual Perceptioncited by 266
  3. 2021Multi-Grained Vision Language Pre-Training: Aligning Texts with Visual Conceptscited by 255
  4. 2023GRES: Generalized Referring Expression Segmentationcited by 252
  5. 2021Vision-Language Transformer and Query Generation for Referring Segmentationcited by 239
  6. 2023MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressionscited by 225
  7. 2023What does CLIP know about a red circle? Visual prompt engineering for VLMscited by 215
  8. 2023Scalable 3D Captioning with Pretrained Modelscited by 201
  9. 2021CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Modelscited by 182
  10. 2025Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videoscited by 154
  11. 2023GSVA: Generalized Segmentation via Multimodal Large Language Modelscited by 153
  12. 2023Alpha-CLIP: A CLIP Model Focusing on Wherever You Wantcited by 150
  13. 2023PolyFormer: Referring Image Segmentation as Sequential Polygon Generationcited by 147
  14. 2021Referring Transformer: A One-step Approach to Multi-task Visual Groundingcited by 143
  15. 2022Towards Adversarial Attack on Vision-Language Pre-training Modelscited by 142
  16. 2023Multi3DRefer: Grounding Text Description to Multiple 3D Objectscited by 142
  17. 2021InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referringcited by 140
  18. 2022SeqTR: A Simple yet Universal Network for Visual Groundingcited by 138
  19. 2022Multi-View Transformer for 3D Visual Groundingcited by 138
  20. 2024OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understandingcited by 126
  21. 2021SAT: 2D Semantics Assisted Training for 3D Visual Groundingcited by 124
  22. 2021UniTAB: Unifying Text and Box Outputs for Grounded Vision-Language Modelingcited by 122
  23. 2022RSVG: Exploring Data and Models for Visual Grounding on Remote Sensing Datacited by 121
  24. 2022Language Conditioned Spatial Relation Reasoning for 3D Object Groundingcited by 121
  25. 2022ReCLIP: A Strong Zero-Shot Baseline for Referring Expression Comprehensioncited by 119
  26. 2022Vision-Language Pre-training: Basics, Recent Advances, and Future Trendscited by 117
  27. 2021Encoder Fusion Network with Co-Attention Embedding for Referring Image Segmentationcited by 111
  28. 2022VLT: Vision-Language Transformer and Query Generation for Referring Segmentationcited by 110
  29. 20223D-SPS: Single-Stage 3D Visual Grounding via Referred Point Progressive Selectioncited by 109
  30. 2022Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbonecited by 109
  31. 2021ArtEmis: Affective Language for Visual Artcited by 103
  32. 2021Cross-Modal Progressive Comprehension for Referring Segmentationcited by 102
  33. 2024PSALM: Pixelwise SegmentAtion with Large Multi-Modal Modelcited by 100
  34. 2021Locate then Segment: A Strong Pipeline for Referring Image Segmentationcited by 98
  35. 2022TubeDETR: Spatio-Temporal Video Grounding with Transformerscited by 97
  36. 2022Exploiting Unlabeled Data with Vision and Language Models for Object Detectioncited by 97
  37. 2024Ferret-v2: An Improved Baseline for Referring and Grounding with Large Language Modelscited by 97
  38. 2021VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluationcited by 95

50 most cited of 427 All 427 (JSON)