# LXMERT Model Compression for Visual Question Answering

arXiv: https://arxiv.org/abs/2310.15325 · HTML: https://mlsanity.com/paper/2310.15325/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 9 references that are in the map.
- First posted on arXiv 23 Oct 2023.

## Builds on (most cited first, 7 of 7)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [Compressing Visual-linguistic Model via Knowledge Distillation](https://mlsanity.com/paper/2104.02096/) (2021)
- [Playing Lottery Tickets with Vision and Language](https://mlsanity.com/paper/2104.11832/) (2021)

## Built on by (most cited first, 1 of 1)

- [Opportunities and challenges in the application of large artificial intelligence models in radiology](https://mlsanity.com/paper/2403.16112/) (2024)

## Abstract (arXiv)

Large-scale pretrained models such as LXMERT are becoming popular for learning cross-modal representations on text-image pairs for vision-language tasks. According to the lottery ticket hypothesis, NLP and computer vision models contain smaller subnetworks capable of being trained in isolation to full performance. In this paper, we combine these observations to evaluate whether such trainable subnetworks exist in LXMERT when fine-tuned on the VQA task. In addition, we perform a model size cost-benefit analysis by investigating how much pruning can be done without significant loss in accuracy. Our experiment results demonstrate that LXMERT can be effectively pruned by 40%-60% in size with 3% loss in accuracy.