mlsanityData to 14 Oct 2026

Fast, Diverse and Accurate Image Captioning Guided By Part-of-Speech

arXiv 1805.12589 ↗First posted 31 May 2018cs.CVcited by 55

Not on the map. Why?

Abstract (arXiv)

Image captioning is an ambiguous problem, with many suitable captions for an image. To address ambiguity, beam search is the de facto method for sampling multiple captions. However, beam search is computationally expensive and known to produce generic captions. To address this concern, some variational auto-encoder (VAE) and generative adversarial net (GAN) based methods have been proposed. Though diverse, GAN and VAE are less accurate. In this paper, we first predict a meaningful summary of the image, then generate the caption based on that summary. We use part-of-speech as summaries, since our summary should drive caption generation. We achieve the trifecta: (1) High accuracy for the diverse captions as evaluated by standard captioning metrics and user studies; (2) Faster computation of diverse captions compared to beam search and diverse beam search; and (3) High diversity as evaluated by counting novel sentences, distinct n-grams and mutual overlap (i.e., mBleu-4) scores.

Key facts. Cited by 55 papers in the map (data through 14 Oct 2026). First posted on arXiv 31 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 16; 2022: 15; 2023: 13; 2024: 7; 2025: 4.

55 papers.

Sort:

By area:

  1. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 450
  2. 2021From Show to Tell: A Survey on Deep Learning-based Image Captioningcited by 138
  3. 2023TESS: Text-to-Text Self-Conditioned Simplex Diffusioncited by 65
  4. 2022SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformerscited by 64
  5. 2023CodeLMSec Benchmark: Systematically Evaluating and Finding Security Vulnerabilities in Black-Box Code Language Modelscited by 55
  6. 2021Human-like Controllable Image Captioning with Verb-specific Semantic Rolescited by 41
  7. 2023LayoutDiffusion: Improving Graphic Layout Generation by Discrete Diffusion Probabilistic Modelscited by 41
  8. 2022Empowering Diffusion Models on the Embedding Space for Text Generationcited by 40
  9. 2023ConZIC: Controllable Zero-shot Image Captioning by Sampling-Based Polishingcited by 33
  10. 2022SceneTrilogy: On Human Scene-Sketch and its Complementarity with Photo and Textcited by 30
  11. 2021Consensus Graph Representation Learning for Better Grounded Image Captioningcited by 28
  12. 2023Multi-modal Machine Learning in Engineering Design: A Review and Future Directionscited by 27
38 more
  1. 2021Towards Accurate Text-based Image Captioning with Content Diversity Explorationcited by 25
  2. 2024Detecting Subtle Differences between Human and Model Languages Using Spectrum of Relative Likelihoodcited by 24
  3. 2023IC3: Image Captioning by Committee Consensuscited by 20
  4. 2022Contextual Modeling for 3D Dense Captioning on Point Cloudscited by 18
  5. 2023Generating Visual Spatial Description via Holistic 3D Scene Understandingcited by 16
  6. 2022Diverse Text Generation via Variational Encoder-Decoder Models with Gaussian Process Priorscited by 13
  7. 2021The Role of Syntactic Planning in Compositional Image Captioningcited by 12
  8. 2022Learning Distinct and Representative Styles for Image Captioningcited by 12
  9. 2023Learning Combinatorial Prompts for Universal Controllable Image Captioningcited by 12
  10. 2023Parts of Speech-Grounded Subspaces in Vision-Language Modelscited by 11
  11. 2021Question-controlled Text-aware Image Captioningcited by 9
  12. 2021Object-Centric Unsupervised Image Captioningcited by 9
  13. 2021Controllable Video Captioning with an Exemplar Sentencecited by 9
  14. 2021LSD-StructureNet: Modeling Levels of Structural Detail in 3D Part Hierarchiescited by 8
  15. 2025Smoothie: Smoothing Diffusion on Token Embeddings for Text Generationcited by 8
  16. 2022Image Captioning via Compact Bidirectional Architecturecited by 7
  17. 2022On Distinctive Image Captioning via Comparing and Reweightingcited by 7
  18. 2021A Comprehensive Review of the Video-to-Text Problemcited by 5
  19. 2023InfoDiffusion: Information Entropy Aware Diffusion Process for Non-Autoregressive Text Generationcited by 5
  20. 2022FlowEval: A Consensus-Based Dialogue Evaluation Framework Using Segment Act Flowscited by 4
  21. 2022CLID: Controlled-Length Image Descriptions with Limited Datacited by 4
  22. 2023Set Prediction Guided by Semantic Concepts for Diverse Video Captioningcited by 4
  23. 2022Self-directed Machine Learningcited by 3
  24. 2022Diverse Image Captioning with Grounded Stylecited by 3
  25. 2024Image Generative Semantic Communication with Multi-Modal Similarity Estimation for Resource-Limited Networkscited by 3
  26. 2024Harnessing the Power of LLMs: Automating Unit Test Generation for High-Performance Computingcited by 3
  27. 2025SGDiff: Scene Graph Guided Diffusion Model for Image Collaborative SegCaptioningcited by 3
  28. 2021Diversifying Neural Text Generation with Part-of-Speech Guided Softmax and Samplingcited by 2
  29. 2021Self-Annotated Training for Controllable Image Captioningcited by 2
  30. 2021Syntax Customized Video Captioning by Imitating Exemplar Sentencescited by 2
  31. 2022Belief Revision based Caption Re-ranker with Visual Semantic Informationcited by 2
  32. 2023Visual Semantic Relatedness Dataset for Image Captioningcited by 2
  33. 2024Fine-grained length controllable video captioning with ordinal embeddingscited by 2
  34. 2025Learning Energy-based Variational Latent Prior for VAEscited by 2
  35. 2021Macroscopic Control of Text Generation for Image Captioningcited by 1
  36. 2021There Once Was a Really Bad Poet, It Was Automated but You Didn't Know Itcited by 1
  37. 2021Goal-driven text descriptions for imagescited by 1
  38. 2023GlyphDiffusion: Text Generation as Image Generationcited by 1

50 most cited of 55 All 55 (JSON)