FormLM: Recommending Creation Ideas for Online Forms by Modelling Semantic and Structural Information
Not on the map. Why?
Abstract (arXiv)
Online forms are widely used to collect data from human and have a multi-billion market. Many software products provide online services for creating semi-structured forms where questions and descriptions are organized by pre-defined structures. However, the design and creation process of forms is still tedious and requires expert knowledge. To assist form designers, in this work we present FormLM to model online forms (by enhancing pre-trained language model with form structural information) and recommend form creation ideas (including question / options recommendations and block type suggestion). For model training and evaluation, we collect the first public online form dataset with 62K online forms. Experiment results show that FormLM significantly outperforms general-purpose language models on all tasks, with an improvement by 4.71 on Question Recommendation and 10.6 on Block Type Suggestion in terms of ROUGE-1 and Macro-F1, respectively.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 29 of its 56 references that are in the map. First posted on arXiv 10 Nov 2022.
Builds on
29 of 56 references.
References · 29
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
- 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
- 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
- 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
- 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
- 2021Ethical and social risks of harm from Language Modelscited by 966
- 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 834
- 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 528
- 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 434
- 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 424
17 more
- 2020LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understandingcited by 408
- 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 398
- 2021DocFormer: End-to-End Transformer for Document Understandingcited by 256
- 2021TAPEX: Table Pre-training via Learning a Neural SQL Executorcited by 235
- 2019BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysiscited by 176
- 2019Hierarchical Transformers for Multi-Document Summarizationcited by 132
- 2021TABBIE: Pretrained Representations of Tabular Datacited by 111
- 2022LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understandingcited by 107
- 2020TUTA: Tree-based Transformers for Generally Structured Table Pre-trainingcited by 100
- 2021StructuralLM: Structural Pre-training for Form Understandingcited by 91
- 2022TableFormer: Robust Transformer Modeling for Table-Text Encodingcited by 91
- 2021MATE: Multi-view Attention for Table Transformer Efficiencycited by 54
- 2021MarkupLM: Pre-training of Text and Markup Language for Visually-rich Document Understandingcited by 45
- 2021SADGA: Structure-Aware Dual Graph Aggregation Network for Text-to-SQLcited by 43
- 2022WebFormer: The Web-page Transformer for Structure Information Extractioncited by 34
- 2019IMHO Fine-Tuning Improves Claim Detectioncited by 27
- 2021Form2Seq : A Framework for Higher-Order Form Structure Extractioncited by 16
Built on by
Citing papers per year: 2023: 3.
3 papers.