mlsanityData to 14 Oct 2026

CLID: Controlled-Length Image Descriptions with Limited Data

arXiv 2211.14835 ↗First posted 27 Nov 2022cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Controllable image captioning models generate human-like image descriptions, enabling some kind of control over the generated captions. This paper focuses on controlling the caption length, i.e. a short and concise description or a long and detailed one. Since existing image captioning datasets contain mostly short captions, generating long captions is challenging. To address the shortage of long training examples, we propose to enrich the dataset with varying-length self-generated captions. These, however, might be of varying quality and are thus unsuitable for conventional training. We introduce a novel training strategy that selects the data points to be used at different times during the training. Our method dramatically improves the length-control abilities, while exhibiting SoTA performance in terms of caption quality. Our approach is general and is shown to be applicable also to paragraph generation.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 60 of its 88 references that are in the map. First posted on arXiv 27 Nov 2022.

Builds on

60 of 88 references.

References · 60

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  4. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  5. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
  6. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  7. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
  8. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,362
  9. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,312
  10. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
  11. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 997
  12. 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 968
38 more
  1. 2014Show and Tell: A Neural Image Caption Generatorcited by 879
  2. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 824
  3. 2014Long-term Recurrent Convolutional Networks for Visual Recognition and Descriptioncited by 484
  4. 2017Scene Graph Generation by Iterative Message Passingcited by 462
  5. 2017Neural Motifs: Scene Graph Parsing with Global Contextcited by 414
  6. 2020Unbiased Scene Graph Generation from Biased Trainingcited by 356
  7. 2019Meshed-Memory Transformer for Image Captioningcited by 343
  8. 2018Image Generation from Scene Graphscited by 324
  9. 2018Graph R-CNN for Scene Graph Generationcited by 298
  10. 2018Auto-Encoding Scene Graphs for Image Captioningcited by 255
  11. 2018Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noisecited by 216
  12. 2019Image Captioning: Transforming Objects into Wordscited by 171
  13. 2017Learning from Noisy Labels with Distillationcited by 161
  14. 2019Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Ratescited by 142
  15. 2018Learning Semantic Segmentation from Synthetic Data: A Geometrically Guided Input-Output Adaptation Approachcited by 119
  16. 2017Learning From Noisy Large-Scale Datasets With Minimal Supervisioncited by 115
  17. 2018Neural Baby Talkcited by 107
  18. 2016A Hierarchical Approach for Generating Descriptive Image Paragraphscited by 104
  19. 2018Explainable and Explicit Visual Reasoning over Scene Graphscited by 103
  20. 2019Graphical Contrastive Losses for Scene Graph Parsingcited by 100
  21. 2020Say As You Wish: Fine-grained Control of Image Caption Generation with Abstract Scene Graphscited by 87
  22. 2019Unpaired Image Captioning via Scene Graph Alignmentscited by 82
  23. 2019Cross-modal Scene Graph Matching for Relationship-aware Image-Text Retrievalcited by 66
  24. 2018Show, Control and Tell: A Framework for Generating Controllable and Grounded Captionscited by 65
  25. 2019Learning from Synthetic Animalscited by 65
  26. 2019Curriculum Learning for Domain Adaptation in Neural Machine Translationcited by 59
  27. 2018Fast, Diverse and Accurate Image Captioning Guided By Part-of-Speechcited by 55
  28. 2020Comprehensive Image Captioning via Scene Graph Decompositioncited by 52
  29. 2018EML-NET:An Expandable Multi-Layer NETwork for Saliency Predictioncited by 51
  30. 2018Denoising Neural Machine Translation Training with Trusted Data and Online Data Selectioncited by 43
  31. 2021Human-like Controllable Image Captioning with Verb-specific Semantic Rolescited by 41
  32. 2020Length-Controllable Image Captioningcited by 36
  33. 2019Dense Relational Captioning: Triple-Stream Networks for Relationship-Based Captioningcited by 35
  34. 2021In Defense of Scene Graphs for Image Captioningcited by 31
  35. 2019Context-Aware Visual Policy Network for Fine-Grained Image Captioningcited by 29
  36. 2018SemStyle: Learning to Generate Stylised Image Captions using Unaligned Textcited by 27
  37. 2018"Factual" or "Emotional": Stylized Image Captioning with Adaptive Learning and Attentioncited by 26
  38. 2019Generating Natural Language Explanations for Visual Question Answering using Scene Graphs and Visual Attentioncited by 26

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 0; 2026: 1.

4 papers.

  1. 2024CIC-BART-SSA: Controllable Image Captioning with Structured Semantic Augmentationcited by 2
  2. 2024Fine-grained length controllable video captioning with ordinal embeddingscited by 2
  3. 2024Curriculum Learning for Cross-Lingual Data-to-Text Generation With Noisy Datacited by 1
  4. 2026Ready to Speak: Aligning LLMs for TTS-Friendly Text Generationcited by 0