mlsanityData to 14 Oct 2026

Expressing Visual Relationships via Language

arXiv 1906.07689 ↗First posted 18 Jun 2019cs.CL cs.CV cs.LGcited by 49

Not on the map. Why?

Abstract (arXiv)

Describing images with text is a fundamental problem in vision-language research. Current studies in this domain mostly focus on single image captioning. However, in various real applications (e.g., image editing, difference interpretation, and retrieval), generating relational captions for two images, can also be very useful. This important problem has not been explored mostly due to lack of datasets and effective models. To push forward the research in this direction, we first introduce a new language-guided image editing dataset that contains a large number of real image pairs with corresponding editing instructions. We then propose a new relational speaker model based on an encoder-decoder architecture with static relational attention and sequential multi-head attention. We also extend the model with dynamic relational attention, which calculates visual alignment while decoding. Our models are evaluated on our newly collected and two public datasets consisting of image pairs annotated with relationship sentences. Experimental results, based on both automatic and human evaluation, demonstrate that our model outperforms all baselines and existing methods on all the datasets.

Key facts. Cited by 49 papers in the map (data through 14 Oct 2026). First posted on arXiv 18 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 5; 2023: 7; 2024: 14; 2025: 11; 2026: 9.

49 papers.

Sort:

By area:

  1. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  2. 2023Guiding Instruction-based Image Editing via Multimodal Large Language Modelscited by 185
  3. 2024MileBench: Benchmarking MLLMs in Long Contextcited by 88
  4. 2023Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructionscited by 77
  5. 2024Towards Semantic Equivalence of Tokenization in Multimodal LLMcited by 45
  6. 2022Image Difference Captioning with Pre-training and Contrastive Learningcited by 39
  7. 2023Visual Instruction Tuning with Polite Flamingocited by 38
  8. 2024Learning Action and Reasoning-Centric Image Editing from Videos and Simulationscited by 32
  9. 2024Auto-Encoding Morph-Tokens for Multimodal LLMcited by 30
  10. 2025Generative Multimodal Pretraining with Discrete Diffusion Timestep Tokenscited by 28
  11. 2022CLIP4IDC: CLIP for Image Difference Captioningcited by 25
  12. 2023Towards Generic Image Manipulation Detection with Weakly-Supervised Self-Consistency Learningcited by 23
37 more
  1. 2023Self-supervised Cross-view Representation Reconstruction for Change Captioningcited by 20
  2. 2024Img-Diff: Contrastive Data Synthesis for Multimodal Large Language Modelscited by 17
  3. 2023Neighborhood Contrastive Transformer for Change Captioningcited by 16
  4. 2024OneDiff: A Generalist Model for Image Difference Captioningcited by 15
  5. 2025REALEDIT: Reddit Edits As a Large-scale Empirical Dataset for Image Transformationscited by 15
  6. 2025TrimTokenator-LC: Towards Adaptive Visual Token Pruning for Large Multimodal Models with Long Contextscited by 14
  7. 2024CCExpert: Advancing MLLM Capability in Remote Sensing Change Captioning with Difference-Aware Integration and a Foundational Datasetcited by 13
  8. 2021FixMyPose: Pose Correctional Captioning and Retrievalcited by 12
  9. 2021R3Net:Relation-embedded Representation Reconstruction Network for Change Captioningcited by 12
  10. 2024VIXEN: Visual Text Comparison Network for Image Difference Captioningcited by 12
  11. 2025OmniDiff: A Comprehensive Benchmark for Fine-grained Image Difference Captioningcited by 12
  12. 2025MadaKV: Adaptive Modality-Perception KV Cache Eviction for Efficient Multimodal Long-Context Inferencecited by 12
  13. 2021L2C: Describing Visual Differences Needs Semantic Understanding of Individualscited by 8
  14. 2022Rethinking the Reference-based Distinctive Image Captioningcited by 8
  15. 2025Slot-MLLM: Object-Centric Visual Tokenization for Multimodal LLMcited by 7
  16. 2024Distractors-Immune Representation Learning with Cross-modal Contrastive Regularization for Change Captioningcited by 6
  17. 2025FlowMM: Cross-Modal Information Flow Guided KV Cache Merging for Efficient Multimodal Context Inferencecited by 6
  18. 2025Understanding Generative AI Capabilities in Everyday Image Editing Taskscited by 5
  19. 2023Improving Reference-based Distinctive Image Captioning with Contrastive Rewardscited by 4
  20. 2024Generative Visual Instruction Tuningcited by 4
  21. 2025Breaking Language Barriers in Visual Language Models via Multilingual Textual Regularizationcited by 4
  22. 2022CAISE: Conversational Agent for Image Search and Editingcited by 3
  23. 2024Reframing Image Difference Captioning with BLIP2IDC and Synthetic Augmentationcited by 3
  24. 2024TAB: Transformer Attention Bottlenecks enable User Intervention and Debugging in Vision-Language Modelscited by 2
  25. 2025Instructify: Demystifying Metadata to Visual Instruction Tuning Data Conversioncited by 2
  26. 2026SPD-Faith Bench: Diagnosing and Improving Faithfulness in Chain-of-Thought for Multimodal Large Language Modelscited by 2
  27. 2024Semi-supervised Chinese Poem-to-Painting Generation via Cycle-consistent Adversarial Networkscited by 1
  28. 2025Image Difference Grounding with Natural Languagecited by 1
  29. 2026TABED: Test-Time Adaptive Ensemble Drafting for Robust Speculative Decoding in LVLMscited by 1
  30. 2026Imagine How To Change: Explicit Procedure Modeling for Change Captioningcited by 1
  31. 2026DiffCap-Bench: A Comprehensive, Challenging, Robust Benchmark for Image Difference Captioningcited by 1
  32. 2022Spot the Difference: A Cooperative Object-Referring Game in Non-Perfectly Co-Observable Scenecited by 0
  33. 2026C3-Bench: A Context-Aware Change Captioning Benchmarkcited by 0
  34. 2026DiCoBench: Benchmarking Multi-Image Fine-Grained Perception via Differential and Commonality Visual Cuescited by 0
  35. 2026Beyond Pixel Diffs: Benchmarking Image Change Captioning for Web UI Visual Regression Testingcited by 0
  36. 2026Different Changes Require Different Reasoning: Change-Type-Specialized Experts for Robust Change Captioningcited by 0
  37. 2026MinCU: A Fine-Grained Benchmark for Grounded Minimal-Change Understanding in Image Pairscited by 0