# TILFA: A Unified Framework for Text, Image, and Layout Fusion in Argument Mining

arXiv: https://arxiv.org/abs/2310.05210 · HTML: https://mlsanity.com/paper/2310.05210/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 32 references that are in the map.
- First posted on arXiv 8 Oct 2023.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://mlsanity.com/paper/2204.08387/) (2022)
- [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://mlsanity.com/paper/2012.14740/) (2020)
- [Enhanced LSTM for Natural Language Inference](https://mlsanity.com/paper/1609.06038/) (2016)
- [A Survey on Data Augmentation for Text Classification](https://mlsanity.com/paper/2107.03158/) (2021)
- [A Comprehensive Survey of Image Augmentation Techniques for Deep Learning](https://mlsanity.com/paper/2205.01491/) (2022)
- [Graph Convolution for Multimodal Information Extraction from Visually Rich Documents](https://mlsanity.com/paper/1903.11279/) (2019)
- [GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge](https://mlsanity.com/paper/1908.07245/) (2019)
- [Argumentation Mining in User-Generated Web Discourse](https://mlsanity.com/paper/1601.02403/) (2016)
- [Is ChatGPT a Good Causal Reasoner? A Comprehensive Evaluation](https://mlsanity.com/paper/2305.07375/) (2023)
- [CAR: Conceptualization-Augmented Reasoner for Zero-Shot Commonsense Question Answering](https://mlsanity.com/paper/2305.14869/) (2023)
- [CAT: A Contextualized Conceptualization and Instantiation Framework for Commonsense Reasoning](https://mlsanity.com/paper/2305.04808/) (2023)
- [COLA: Contextualized Commonsense Causal Reasoning from the Causal Inference Perspective](https://mlsanity.com/paper/2305.05191/) (2023)
- [Incomplete Multi-View Multi-Label Learning via Label-Guided Masked View- and Category-Aware Transformers](https://mlsanity.com/paper/2303.07180/) (2023)
- [ROCK: Causal Inference Principles for Reasoning about Commonsense Causality](https://mlsanity.com/paper/2202.00436/) (2022)
- [Getting Sick After Seeing a Doctor? Diagnosing and Mitigating Knowledge Conflicts in Event Temporal Reasoning](https://mlsanity.com/paper/2305.14970/) (2023)
- [ImageArg: A Multi-modal Tweet Dataset for Image Persuasiveness Mining](https://mlsanity.com/paper/2209.06416/) (2022)
- [A Multi-Modal Context Reasoning Approach for Conditional Inference on Joint Textual and Visual Clues](https://mlsanity.com/paper/2305.04530/) (2023)
- [SubeventWriter: Iterative Sub-event Sequence Generation with Coherence Controller](https://mlsanity.com/paper/2210.06694/) (2022)
- [Chunk-aware Alignment and Lexical Constraint for Visual Entailment with Natural Language Explanations](https://mlsanity.com/paper/2207.11401/) (2022)
- [Global Constraints with Prompting for Zero-Shot Event Argument Classification](https://mlsanity.com/paper/2302.04459/) (2023)

## Built on by (most cited first, 3 of 3)

- [MIND: Multimodal Shopping Intention Distillation from Large Vision-language Models for E-commerce Purchase Understanding](https://mlsanity.com/paper/2406.10701/) (2024)
- [ComparisonQA: Evaluating Factuality Robustness of LLMs Through Knowledge Frequency Control and Uncertainty](https://mlsanity.com/paper/2412.20251/) (2024)
- [KNOWCOMP POKEMON Team at DialAM-2024: A Two-Stage Pipeline for Detecting Relations in Dialogical Argument Mining](https://mlsanity.com/paper/2407.19740/) (2024)

## Abstract (arXiv)

A main goal of Argument Mining (AM) is to analyze an author's stance. Unlike previous AM datasets focusing only on text, the shared task at the 10th Workshop on Argument Mining introduces a dataset including both text and images. Importantly, these images contain both visual elements and optical characters. Our new framework, TILFA (A Unified Framework for Text, Image, and Layout Fusion in Argument Mining), is designed to handle this mixed data. It excels at not only understanding text but also detecting optical characters and recognizing layout details in images. Our model significantly outperforms existing baselines, earning our team, KnowComp, the 1st place in the leaderboard of Argumentative Stance Classification subtask in this shared task.