mlsanityData to 14 Oct 2026

End-to-End Attention-based Image Captioning

arXiv 2104.14721 ↗First posted 30 Apr 2021cs.CV cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

In this paper, we address the problem of image captioning specifically for molecular translation where the result would be a predicted chemical notation in InChI format for a given molecular structure. Current approaches mainly follow rule-based or CNN+RNN based methodology. However, they seem to underperform on noisy images and images with small number of distinguishable features. To overcome this, we propose an end-to-end transformer model. When compared to attention-based techniques, our proposed model outperforms on molecular datasets.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 16 of its 25 references that are in the map. First posted on arXiv 30 Apr 2021.

Builds on

16 of 25 references.

References · 16

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  4. 2020Longformer: The Long-Document Transformercited by 3,413
  5. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
  6. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,369
  7. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,312
  8. 2016Self-critical Sequence Training for Image Captioningcited by 547
  9. 2018A Comprehensive Survey of Deep Learning for Image Captioningcited by 214
  10. 2016Show and Tell: Lessons learned from the 2015 MSCOCO Image Captioning Challengecited by 116
  11. 2014Deep Captioning with Multimodal Recurrent Neural Networks (m-RNN)cited by 94
  12. 2017SMILES2Vec: An Interpretable General-Purpose Deep Neural Network for Predicting Chemical Propertiescited by 30
4 more
  1. 2018Molecular Structure Extraction From Documents Using Deep Learningcited by 25
  2. 2017Chemception: A Deep Neural Network with Minimal Chemistry Knowledge Matches the Performance of Expert-developed QSAR/QSPR Modelscited by 19
  3. 2020ChemGrapher: Optical Graph Recognition of Chemical Compounds by Deep Learningcited by 18
  4. 2018CheMixNet: Mixed DNN Architectures for Predicting Chemical Properties using Multiple Molecular Representationscited by 8

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 1; 2023: 1; 2024: 1.

4 papers.

  1. 2022Translation between Molecules and Natural Languagecited by 177
  2. 2021Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videoscited by 141
  3. 2023MolGrapher: Graph-based Visual Recognition of Chemical Structurescited by 24
  4. 2024RFL: Simplifying Chemical Structure Recognition with Ring-Free Languagecited by 1