mlsanityData to 7 Oct 2026

Rethinking Referring Object Removal

arXiv 2403.09128 ↗First posted 14 Mar 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Referring object removal refers to removing the specific object in an image referred by natural language expressions and filling the missing region with reasonable semantics. To address this task, we construct the ComCOCO, a synthetic dataset consisting of 136,495 referring expressions for 34,615 objects in 23,951 image pairs. Each pair contains an image with referring expressions and the ground truth after elimination. We further propose an end-to-end syntax-aware hybrid mapping network with an encoding-decoding structure. Linguistic features are hierarchically extracted at the syntactic level and fused in the downsampling process of visual features with multi-head attention. The feature-aligned pyramid network is leveraged to generate segmentation masks and replace internal pixels with region affinity learned from external semantics in high-level feature maps. Extensive experiments demonstrate that our model outperforms diffusion models and two-stage methods which process the segmentation and inpainting task separately by a significant margin.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 41 of its 52 references that are in the map. First posted on arXiv 14 Mar 2024.

Builds on

41 of 52 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  6. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  7. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  8. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  9. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  10. 2017Deformable Convolutional Networkscited by 1,481
  11. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  12. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
29 more
  1. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  2. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
  3. 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,128
  4. 2016Modeling Context in Referring Expressionscited by 1,054
  5. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
  6. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 730
  7. 2016Neural Architectures for Named Entity Recognitioncited by 548
  8. 2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
  9. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 347
  10. 2018Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomizationcited by 292
  11. 2023DiffIR: Efficient Diffusion Model for Image Restorationcited by 253
  12. 2021Vision-Language Transformer and Query Generation for Referring Segmentationcited by 239
  13. 2020Multi-task Collaborative Network for Joint Referring Expression Comprehension and Segmentationcited by 204
  14. 2023SmartEdit: Exploring Complex Instruction-based Image Editing with Multimodal Large Language Modelscited by 174
  15. 2023PolyFormer: Referring Image Segmentation as Sequential Polygon Generationcited by 147
  16. 2020PhraseCut: Language-based Image Segmentation in the Wildcited by 116
  17. 2022ReSTR: Convolution-free Referring Image Segmentation Using Transformerscited by 112
  18. 2021Encoder Fusion Network with Co-Attention Embedding for Referring Image Segmentationcited by 111
  19. 2019Learning Pyramid-Context Encoder Network for High-Quality Image Inpaintingcited by 94
  20. 2022Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encodingcited by 83
  21. 2021FaPN: Feature-aligned Pyramid Network for Dense Image Predictioncited by 81
  22. 2021Image Inpainting via Conditional Texture and Structure Dual Generationcited by 66
  23. 2022Harmonizer: Learning to Perform White-Box Image and Video Harmonizationcited by 61
  24. 2017Contextual-based Image Inpainting: Infer, Match, and Translatecited by 50
  25. 2022Reduce Information Loss in Transformers for Pluralistic Image Inpaintingcited by 36
  26. 2021Image Inpainting with External-internal Learning and Monochromic Bottleneckcited by 21
  27. 2022Learning Object Placement via Dual-path Graph Completioncited by 20
  28. 2023Meta Compositional Referring Expression Segmentationcited by 18
  29. 2023Semantics-Aware Dynamic Localization and Refinement for Referring Image Segmentationcited by 16

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025FLORA: Formal Language Model Enables Robust Training-free Zero-shot Object Referring Analysis
  2. 2025Deformable Attentive Visual Enhancement for Referring Segmentation Using Vision-Language Model
  3. 2025LGD: Leveraging Generative Descriptions for Zero-Shot Referring Image Segmentation
  4. 2025Towards Unified Referring Expression Segmentation Across Omni-Level Visual Target Granularities
  5. 2026Towards Open-World Referring Expression Comprehension: A Benchmark with Training-free Multi-task Consistency Checker
  6. 2023Ref-Diff: Zero-shot Referring Image Segmentation with Generative Models
  7. 2025Rex-Thinker: Grounded Object Referring via Chain-of-Thought Reasoning
  8. 2026Interpretable Zero-shot Referring Expression Comprehension with Query-driven Scene Graphs