Relational Future Captioning Model for Explaining Likely Collisions in Daily Tasks
Not on the map. Why?
Abstract (arXiv)
Domestic service robots that support daily tasks are a promising solution for elderly or disabled people. It is crucial for domestic service robots to explain the collision risk before they perform actions. In this paper, our aim is to generate a caption about a future event. We propose the Relational Future Captioning Model (RFCM), a crossmodal language generation model for the future captioning task. The RFCM has the Relational Self-Attention Encoder to extract the relationships between events more effectively than the conventional self-attention in transformers. We conducted comparison experiments, and the results show the RFCM outperforms a baseline method on two datasets.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 21 of its 32 references that are in the map. First posted on arXiv 19 Jul 2022.
Builds on
21 of 32 references.
References · 21
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
- 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
- 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
- 2017Dense-Captioning Events in Videoscited by 789
- 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 702
- 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 616
- 2019End-to-End Learning of Visual Representations from Uncurated Instructional Videoscited by 506
- 2020UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generationcited by 258
- 2018A Comprehensive Survey of Deep Learning for Image Captioningcited by 213
- 2020COOT: Cooperative Hierarchical Transformer for Video-Text Representation Learningcited by 100
9 more
- 2020MART: Memory-Augmented Recurrent Transformer for Coherent Video Paragraph Captioningcited by 79
- 2021Greedy Hierarchical Variational Autoencoders for Large-Scale Video Predictioncited by 75
- 2020What is More Likely to Happen Next? Video-and-Language Future Event Predictioncited by 61
- 2019Streamlined Dense Video Captioningcited by 57
- 2017Video Captioning via Hierarchical Reinforcement Learningcited by 47
- 2021Relational Self-Attention: What's Missing in Attention for Video Understandingcited by 15
- 2021Case Relation Transformer: A Crossmodal Language Generation Model for Fetching Instructionscited by 4
- 2019Prediction and Description of Near-Future Activities in Videocited by 3
- 2021Predicting and Attending to Damaging Collisions for Placing Everyday Objects in Photo-Realistic Simulationscited by 2
Built on by
Citing papers per year: 2024: 2.
2 papers.