# DenseCap: Fully Convolutional Localization Networks for Dense Captioning

arXiv: https://arxiv.org/abs/1511.07571 · HTML: https://mlsanity.com/paper/1511.07571/

## Key facts

- Cited by 176 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Nov 2015.

## Built on by (most cited first, 30 of 176)

- [Unifying Vision-and-Language Tasks via Text Generation](https://mlsanity.com/paper/2102.02779/) (2021)
- [Pre-Trained Models: Past, Present and Future](https://mlsanity.com/paper/2106.07139/) (2021)
- [GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest](https://mlsanity.com/paper/2307.03601/) (2023)
- [VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioning](https://mlsanity.com/paper/2102.10407/) (2021)
- [Dense Text-to-Image Generation with Attention Modulation](https://mlsanity.com/paper/2308.12964/) (2023)
- [Interactive and Explainable Region-guided Radiology Report Generation](https://mlsanity.com/paper/2304.08295/) (2023)
- [From Show to Tell: A Survey on Deep Learning-based Image Captioning](https://mlsanity.com/paper/2107.06912/) (2021)
- [Bottom Up Top Down Detection Transformers for Language Grounding in Images and Point Clouds](https://mlsanity.com/paper/2112.08879/) (2021)
- [DRAMA: Joint Risk Localization and Captioning in Driving](https://mlsanity.com/paper/2209.10767/) (2022)
- [GRiT: A Generative Region-to-text Transformer for Object Understanding](https://mlsanity.com/paper/2212.00280/) (2022)
- [Caption Anything: Interactive Image Description with Diverse Multimodal Controls](https://mlsanity.com/paper/2305.02677/) (2023)
- [ClevrTex: A Texture-Rich Benchmark for Unsupervised Multi-Object Segmentation](https://mlsanity.com/paper/2111.10265/) (2021)
- [Language Models Can See: Plugging Visual Controls in Text Generation](https://mlsanity.com/paper/2205.02655/) (2022)
- [Expressive Text-to-Image Generation with Rich Text](https://mlsanity.com/paper/2304.06720/) (2023)
- [Make-A-Story: Visual Memory Conditioned Consistent Story Generation](https://mlsanity.com/paper/2211.13319/) (2022)
- [Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey](https://mlsanity.com/paper/2411.17558/) (2024)
- [X-Trans2Cap: Cross-Modal Knowledge Transfer using Transformer for 3D Dense Captioning](https://mlsanity.com/paper/2203.00843/) (2022)
- [FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusion](https://mlsanity.com/paper/2402.03226/) (2024)
- [Rank2Tell: A Multimodal Driving Dataset for Joint Importance Ranking and Reasoning](https://mlsanity.com/paper/2309.06597/) (2023)
- [SoccerNet-Caption: Dense Video Captioning for Soccer Broadcasts Commentaries](https://mlsanity.com/paper/2304.04565/) (2023)
- [Integrating Visuospatial, Linguistic and Commonsense Structure into Story Visualization](https://mlsanity.com/paper/2110.10834/) (2021)
- [D3Net: A Unified Speaker-Listener Architecture for 3D Dense Captioning and Visual Grounding](https://mlsanity.com/paper/2112.01551/) (2021)
- [DetCLIPv3: Towards Versatile Generative Open-vocabulary Object Detection](https://mlsanity.com/paper/2404.09216/) (2024)
- [FaithScore: Fine-grained Evaluations of Hallucinations in Large Vision-Language Models](https://mlsanity.com/paper/2311.01477/) (2023)
- [Visual Goal-Step Inference using wikiHow](https://mlsanity.com/paper/2104.05845/) (2021)
- [Deep Learning Approaches on Image Captioning: A Review](https://mlsanity.com/paper/2201.12944/) (2022)
- [Image Textualization: An Automatic Framework for Creating Accurate and Detailed Image Descriptions](https://mlsanity.com/paper/2406.07502/) (2024)
- [Spatiality-guided Transformer for 3D Dense Captioning on Point Clouds](https://mlsanity.com/paper/2204.10688/) (2022)
- [The Abduction of Sherlock Holmes: A Dataset for Visual Abductive Reasoning](https://mlsanity.com/paper/2202.04800/) (2022)
- [ContIG: Self-supervised Multimodal Contrastive Learning for Medical Imaging with Genetics](https://mlsanity.com/paper/2111.13424/) (2021)

## Abstract (arXiv)

We introduce the dense captioning task, which requires a computer vision system to both localize and describe salient regions in images in natural language. The dense captioning task generalizes object detection when the descriptions consist of a single word, and Image Captioning when one predicted region covers the full image. To address the localization and description task jointly we propose a Fully Convolutional Localization Network (FCLN) architecture that processes an image with a single, efficient forward pass, requires no external regions proposals, and can be trained end-to-end with a single round of optimization. The architecture is composed of a Convolutional Network, a novel dense localization layer, and Recurrent Neural Network language model that generates the label sequences. We evaluate our network on the Visual Genome dataset, which comprises 94,000 images and 4,100,000 region-grounded captions. We observe both speed and accuracy improvements over baselines based on current state of the art approaches in both generation and retrieval settings.