# Attention Strategies for Multi-Source Sequence-to-Sequence Learning

arXiv: https://arxiv.org/abs/1704.06567 · HTML: https://mlsanity.com/paper/1704.06567/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Apr 2017.

## Built on by (most cited first, 30 of 32)

- [Investigating the Translation Performance of a Large Multilingual Language Model: the Case of BLOOM](https://mlsanity.com/paper/2303.01911/) (2023)
- [Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation](https://mlsanity.com/paper/2105.14462/) (2021)
- [Vision Guided Generative Pre-trained Language Models for Multimodal Abstractive Summarization](https://mlsanity.com/paper/2109.02401/) (2021)
- [On Vision Features in Multimodal Machine Translation](https://mlsanity.com/paper/2203.09173/) (2022)
- [Neural Machine Translation with Phrase-Level Universal Visual Representations](https://mlsanity.com/paper/2203.10299/) (2022)
- [Exploring Better Text Image Translation with Multimodal Codebook](https://mlsanity.com/paper/2305.17415/) (2023)
- [LVP-M3: Language-aware Visual Prompt for Multilingual Multimodal Machine Translation](https://mlsanity.com/paper/2210.15461/) (2022)
- [Cross-lingual Visual Pre-training for Multimodal Machine Translation](https://mlsanity.com/paper/2101.10044/) (2021)
- [See, Hear, Read: Leveraging Multimodality with Guided Attention for Abstractive Text Summarization](https://mlsanity.com/paper/2105.09601/) (2021)
- [Combining Spectral and Self-Supervised Features for Low Resource Speech Recognition and Translation](https://mlsanity.com/paper/2204.02470/) (2022)
- [AFRIDOC-MT: Document-level MT Corpus for African Languages](https://mlsanity.com/paper/2501.06374/) (2025)
- [Product-oriented Machine Translation with Cross-modal Cross-lingual Pre-training](https://mlsanity.com/paper/2108.11119/) (2021)
- [Contrastive Learning for Context-aware Neural Machine TranslationUsing Coreference Information](https://mlsanity.com/paper/2109.05712/) (2021)
- [Attention-based Multi-hypothesis Fusion for Speech Summarization](https://mlsanity.com/paper/2111.08201/) (2021)
- [On Measuring Context Utilization in Document-Level MT Systems](https://mlsanity.com/paper/2402.01404/) (2024)
- [AnyTrans: Translate AnyText in the Image with Large Scale Models](https://mlsanity.com/paper/2406.11432/) (2024)
- [Exploiting Multimodal Reinforcement Learning for Simultaneous Machine Translation](https://mlsanity.com/paper/2102.11387/) (2021)
- [Text is no more Enough! A Benchmark for Profile-based Spoken Language Understanding](https://mlsanity.com/paper/2112.11953/) (2021)
- [Beyond Triplet: Leveraging the Most Data for Multimodal Machine Translation](https://mlsanity.com/paper/2212.10313/) (2022)
- [Fusing Multimodal Signals on Hyper-complex Space for Extreme Abstractive Text Summarization (TL;DR) of Scientific Contents](https://mlsanity.com/paper/2306.13968/) (2023)
- [A Survey on Multi-modal Machine Translation: Tasks, Methods and Challenges](https://mlsanity.com/paper/2405.12669/) (2024)
- [Low-resource Neural Machine Translation with Cross-modal Alignment](https://mlsanity.com/paper/2210.06716/) (2022)
- [BASS: Block-wise Adaptation for Speech Summarization](https://mlsanity.com/paper/2307.08217/) (2023)
- [ParCourE: A Parallel Corpus Explorer for a Massively Multilingual Corpus](https://mlsanity.com/paper/2107.06632/) (2021)
- [Incorporating Probing Signals into Multimodal Machine Translation via Visual Question-Answering Pairs](https://mlsanity.com/paper/2310.17133/) (2023)
- [Visual Cues and Error Correction for Translation Robustness](https://mlsanity.com/paper/2103.07352/) (2021)
- [Supervised Visual Attention for Simultaneous Multimodal Machine Translation](https://mlsanity.com/paper/2201.09324/) (2022)
- [ICAF: Iterative Contrastive Alignment Framework for Multimodal Abstractive Summarization](https://mlsanity.com/paper/2108.05123/) (2021)
- [How effective is Multi-source pivoting for Translation of Low Resource Indian Languages?](https://mlsanity.com/paper/2406.13332/) (2024)
- [CFSum: A Coarse-to-Fine Contribution Network for Multimodal Summarization](https://mlsanity.com/paper/2307.02716/) (2023)

## Abstract (arXiv)

Modeling attention in neural multi-source sequence-to-sequence learning remains a relatively unexplored area, despite its usefulness in tasks that incorporate multiple source languages or modalities. We propose two novel approaches to combine the outputs of attention mechanisms over each source sequence, flat and hierarchical. We compare the proposed methods with existing techniques and present results of systematic evaluation of those methods on the WMT16 Multimodal Translation and Automatic Post-editing tasks. We show that the proposed methods achieve competitive results on both tasks.