mlsanityData to 7 Oct 2026

QueryForm: A Simple Zero-shot Form Entity Query Framework

arXiv 2211.07730 ↗First posted 14 Nov 2022cs.LG cs.AI cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

Zero-shot transfer learning for document understanding is a crucial yet under-investigated scenario to help reduce the high cost involved in annotating document entities. We present a novel query-based framework, QueryForm, that extracts entity values from form-like documents in a zero-shot fashion. QueryForm contains a dual prompting mechanism that composes both the document schema and a specific entity type into a query, which is used to prompt a Transformer model to perform a single entity extraction task. Furthermore, we propose to leverage large-scale query-entity pairs generated from form-like webpages with weak HTML annotations to pre-train QueryForm. By unifying pre-training and fine-tuning into the same query-based framework, QueryForm enables models to learn from structured documents containing various entities and layouts, leading to better generalization to target document types without the need for target-specific training data. QueryForm sets new state-of-the-art average F1 score on both the XFUND (+4.6%~10.1%) and the Payment (+3.2%~9.5%) zero-shot benchmark, with a smaller model size and no additional image input.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 31 of its 44 references that are in the map. First posted on arXiv 14 Nov 2022.

Builds on

31 of 44 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  7. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  8. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  9. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  10. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  11. 2021Learning to Prompt for Continual Learningcited by 938
  12. 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 635
19 more
  1. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 549
  2. 2020LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understandingcited by 406
  3. 2021OCR-free Document Understanding Transformercited by 400
  4. 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 345
  5. 2021DocFormer: End-to-End Transformer for Document Understandingcited by 254
  6. 2020InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Trainingcited by 237
  7. 2021LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understandingcited by 103
  8. 2018Chargrid: Towards Understanding 2D Documentscited by 100
  9. 2022FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extractioncited by 64
  10. 2019BERTgrid: Contextualized Embedding for 2D Document Representation and Understandingcited by 63
  11. 2021HTLM: Hyper-Text Pre-Training and Prompting of Language Modelscited by 58
  12. 2022Exploring the Universal Vulnerability of Prompt-based Learning Paradigmcited by 56
  13. 2020LAMBERT: Layout-Aware (Language) Modeling for information extractioncited by 53
  14. 2017CloudScan - A configuration-free invoice analysis system using recurrent neural networkscited by 33
  15. 2021ROPE: Reading Order Equivariant Positional Encoding for Graph-based Document Information Extractioncited by 22
  16. 2019CUTIE: Learning to Understand Documents with Convolutional Universal Text Information Extractorcited by 21
  17. 2018Unseen Class Discovery in Open-world Classificationcited by 20
  18. 2021Form2Seq : A Framework for Higher-Order Form Structure Extractioncited by 16
  19. 2021Value Retrieval with Arbitrary Queries for Form-like Documentscited by 5

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 3; 2025: 1.

7 papers.

  1. 2023LMDX: Language Model-based Document Information Extraction and Localizationcited by 37
  2. 2024Deep Learning based Visually Rich Document Content Understanding: A Surveycited by 21
  3. 2023FormNetV2: Multimodal Graph Contrastive Learning for Form Document Information Extractioncited by 20
  4. 2025Information Extraction from Visually Rich Documents using LLM-based Organization of Documents into Independent Textual Segmentscited by 9
  5. 2024Unveiling the Deficiencies of Pre-trained Text-and-Layout Models in Real-world Visually-rich Document Information Extractioncited by 3
  6. 2023On Task-personalized Multimodal Few-shot Learning for Visually-rich Document Entity Retrievalcited by 2
  7. 2024Information Extraction from Heterogeneous Documents without Ground Truth Labels using Synthetic Label Generation and Knowledge Distillationcited by 0