CICA: Content-Injected Contrastive Alignment for Zero-Shot Document Image Classification
Not on the map. Why?
Abstract (arXiv)
Zero-shot learning has been extensively investigated in the broader field of visual recognition, attracting significant interest recently. However, the current work on zero-shot learning in document image classification remains scarce. The existing studies either focus exclusively on zero-shot inference, or their evaluation does not align with the established criteria of zero-shot evaluation in the visual recognition domain. We provide a comprehensive document image classification analysis in Zero-Shot Learning (ZSL) and Generalized Zero-Shot Learning (GZSL) settings to address this gap. Our methodology and evaluation align with the established practices of this domain. Additionally, we propose zero-shot splits for the RVL-CDIP dataset. Furthermore, we introduce CICA (pronounced 'ki-ka'), a framework that enhances the zero-shot learning capabilities of CLIP. CICA consists of a novel 'content module' designed to leverage any generic document-related textual information. The discriminative features extracted by this module are aligned with CLIP's text and image features using a novel 'coupled-contrastive' loss. Our module improves CLIP's ZSL top-1 accuracy by 6.7% and GZSL harmonic mean by 24% on the RVL-CDIP dataset. Our module is lightweight and adds only 3.3% more parameters to CLIP. Our work sets the direction for future research in zero-shot document classification.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 25 of its 43 references that are in the map. First posted on arXiv 6 May 2024.
Builds on
25 of 43 references.
References · 25
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
- 2017Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Uglycited by 538
- 2019Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNetcited by 260
- 2015Evaluation of Deep Convolutional Nets for Document Image Classification and Retrievalcited by 213
- 2018Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoderscited by 206
- 2020Attribute Prototype Network for Zero-Shot Learningcited by 160
- 2016An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wildcited by 146
- 2016Latent Embeddings for Zero-shot Classificationcited by 129
- 2017Generalized Zero-Shot Learning via Synthesized Examplescited by 114
13 more
- 2016Synthesized Classifiers for Zero-Shot Learningcited by 113
- 2021Open World Compositional Zero-Shot Learningcited by 101
- 2017A Generative Adversarial Approach for Zero-Shot Learning from Noisy Textscited by 96
- 2019Transferable Contrastive Network for Generalized Zero-Shot Learningcited by 64
- 2022I2MVFormer: Large Language Model Generated Multi-View Document Supervision for Zero-Shot Image Classificationcited by 54
- 2017Cutting the Error by Half: Investigation of Very Deep CNN and Advanced Training Strategies for Document Image Classificationcited by 22
- 2018Document Image Classification with Intra-Domain Transfer Learning and Stacked Generalization of Deep Convolutional Neural Networkscited by 20
- 2019Modular Multimodal Architecture for Document Classificationcited by 15
- 2017Real-Time Document Image Classification using Deep CNN and Extreme Learning Machinescited by 14
- 2017Analysis of Convolutional Neural Networks for Document Image Classificationcited by 13
- 2020Self-Supervised Representation Learning on Document Imagescited by 5
- 2022Contrastive Training Improves Zero-Shot Classification of Semi-structured Documentscited by 3
- 2024Human Pose Descriptions and Subject-Focused Attention for Improved Zero-Shot Transfer in Human-Centric Classification Taskscited by 3
Built on by
Citing papers per year: 2025: 1.
1 paper.