# Alleviating the Burden of Labeling: Sentence Generation by Attention Branch Encoder-Decoder Network

arXiv: https://arxiv.org/abs/2007.04557 · HTML: https://mlsanity.com/paper/2007.04557/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jul 2020.

## Built on by (most cited first, 6 of 6)

- [DENEB: A Hallucination-Robust Automatic Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2409.19255/) (2024)
- [Target-dependent UNITER: A Transformer-Based Multimodal Language Comprehension Model for Domestic Service Robots](https://mlsanity.com/paper/2107.00811/) (2021)
- [Layer-Wise Relevance Propagation with Conservation Property for ResNet](https://mlsanity.com/paper/2407.09115/) (2024)
- [JaSPICE: Automatic Evaluation Metric Using Predicate-Argument Structures for Image Captioning Models](https://mlsanity.com/paper/2311.04192/) (2023)
- [Switching Head-Tail Funnel UNITER for Dual Referring Expression Comprehension with Fetch-and-Carry Tasks](https://mlsanity.com/paper/2307.07166/) (2023)
- [Case Relation Transformer: A Crossmodal Language Generation Model for Fetching Instructions](https://mlsanity.com/paper/2107.00789/) (2021)

## Abstract (arXiv)

Domestic service robots (DSRs) are a promising solution to the shortage of home care workers. However, one of the main limitations of DSRs is their inability to interact naturally through language. Recently, data-driven approaches have been shown to be effective for tackling this limitation; however, they often require large-scale datasets, which is costly. Based on this background, we aim to perform automatic sentence generation of fetching instructions: for example, "Bring me a green tea bottle on the table." This is particularly challenging because appropriate expressions depend on the target object, as well as its surroundings. In this paper, we propose the attention branch encoder--decoder network (ABEN), to generate sentences from visual inputs. Unlike other approaches, the ABEN has multimodal attention branches that use subword-level attention and generate sentences based on subword embeddings. In experiments, we compared the ABEN with a baseline method using four standard metrics in image captioning. Results show that the ABEN outperformed the baseline in terms of these metrics.