# Discrete-continuous Action Space Policy Gradient-based Attention for Image-Text Matching

arXiv: https://arxiv.org/abs/2104.10406 · HTML: https://mlsanity.com/paper/2104.10406/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 38 references that are in the map.
- First posted on arXiv 21 Apr 2021.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [Deep Visual-Semantic Alignments for Generating Image Descriptions](https://mlsanity.com/paper/1412.2306/) (2014)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [Stacked Cross Attention for Image-Text Matching](https://mlsanity.com/paper/1803.08024/) (2018)
- [Visual Semantic Reasoning for Image-Text Matching](https://mlsanity.com/paper/1909.02701/) (2019)
- [Deep Reinforcement Learning in Large Discrete Action Spaces](https://mlsanity.com/paper/1512.07679/) (2015)
- [Dual-Path Convolutional Image-Text Embeddings with Instance Loss](https://mlsanity.com/paper/1711.05535/) (2017)
- [Discretizing Continuous Action Space for On-Policy Optimization](https://mlsanity.com/paper/1901.10500/) (2019)
- [Learning Semantic Concepts and Order for Image and Sentence Matching](https://mlsanity.com/paper/1712.02036/) (2017)
- [Look, Imagine and Match: Improving Textual-Visual Cross-Modal Retrieval with Generative Models](https://mlsanity.com/paper/1711.06420/) (2017)
- [Convolutional Image Captioning](https://mlsanity.com/paper/1711.09151/) (2017)
- [VQS: Linking Segmentations to Questions and Answers for Supervised Attention in VQA and Question-Focused Semantic Segmentation](https://mlsanity.com/paper/1708.04686/) (2017)
- [Instance-aware Image and Sentence Matching with Selective Multimodal LSTM](https://mlsanity.com/paper/1611.05588/) (2016)
- [Neural Machine Translation with Supervised Attention](https://mlsanity.com/paper/1609.04186/) (2016)
- [Linking Image and Text with 2-Way Nets](https://mlsanity.com/paper/1608.07973/) (2016)
- [Leveraging Visual Question Answering for Image-Caption Ranking](https://mlsanity.com/paper/1605.01379/) (2016)

## Built on by (most cited first, 8 of 8)

- [A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspective](https://mlsanity.com/paper/2209.13232/) (2022)
- [Efficient Token-Guided Image-Text Retrieval with Consistent Multimodal Contrastive Training](https://mlsanity.com/paper/2306.08789/) (2023)
- [ELIP: Enhanced Visual-Language Foundation Models for Image Retrieval](https://mlsanity.com/paper/2502.15682/) (2025)
- [Towards Deconfounded Image-Text Matching with Causal Inference](https://mlsanity.com/paper/2408.12292/) (2024)
- [Soft Actor-Critic With Integer Actions](https://mlsanity.com/paper/2109.08512/) (2021)
- [Object-Aware Query Perturbation for Cross-Modal Image-Text Retrieval](https://mlsanity.com/paper/2407.12346/) (2024)
- [Selectively Hard Negative Mining for Alleviating Gradient Vanishing in Image-Text Matching](https://mlsanity.com/paper/2303.00181/) (2023)
- [AdaTriplet-RA: Domain Matching via Adaptive Triplet and Reinforced Attention for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2211.08894/) (2022)

## Abstract (arXiv)

Image-text matching is an important multi-modal task with massive applications. It tries to match the image and the text with similar semantic information. Existing approaches do not explicitly transform the different modalities into a common space. Meanwhile, the attention mechanism which is widely used in image-text matching models does not have supervision. We propose a novel attention scheme which projects the image and text embedding into a common space and optimises the attention weights directly towards the evaluation metrics. The proposed attention scheme can be considered as a kind of supervised attention and requiring no additional annotations. It is trained via a novel Discrete-continuous action space policy gradient algorithm, which is more effective in modelling complex action space than previous continuous action space policy gradient. We evaluate the proposed methods on two widely-used benchmark datasets: Flickr30k and MS-COCO, outperforming the previous approaches by a large margin.