Towards Multi-Modal DBMSs for Seamless Querying of Texts and Tables
Not on the map. Why?
Abstract (arXiv)
In this paper, we propose Multi-Modal Databases (MMDBs), which is a new class of database systems that can seamlessly query text and tables using SQL. To enable seamless querying of textual data using SQL in an MMDB, we propose to extend relational databases with so-called multi-modal operators (MMOps) which are based on the advances of recent large language models such as GPT-3. The main idea of MMOps is that they allow text collections to be treated as tables without the need to manually transform the data. As we show in our evaluation, our MMDB prototype can not only outperform state-of-the-art approaches such as text-to-table in terms of accuracy and performance but it also requires significantly less training data to fine-tune the model for an unseen text collection.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 38 of its 48 references that are in the map. First posted on arXiv 26 Apr 2023.
Builds on
38 of 48 references.
References · 38
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
- 2018Deep contextualized word representationscited by 2,290
- 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
- 2022Survey of Hallucination in Natural Language Generationcited by 2,075
- 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
- 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,296
- 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,043
- 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
26 more
- 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 834
- 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 573
- 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 398
- 2020ToTTo: A Controlled Table-To-Text Generation Datasetcited by 268
- 2017The E2E Dataset: New Challenges For End-to-End Generationcited by 253
- 2017End-to-end Neural Coreference Resolutioncited by 241
- 2021TAPEX: Table Pre-training via Learning a Neural SQL Executorcited by 235
- 2017Challenges in Data-to-Document Generationcited by 226
- 2022Can Foundation Models Wrangle Your Data?cited by 175
- 2020GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsingcited by 171
- 2020Structure-Grounded Pretraining for Text-to-SQLcited by 126
- 2021TABBIE: Pretrained Representations of Tabular Datacited by 111
- 2020TUTA: Tree-based Transformers for Generally Structured Table Pre-trainingcited by 100
- 2020Pre-training via Paraphrasingcited by 92
- 2020Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Trainingcited by 65
- 2021Few-Shot Question Answering by Pretraining Span Selectioncited by 62
- 2021Text-to-Table: A New Way of Information Extractioncited by 47
- 2021GitTables: A Large-Scale Corpus of Relational Tablescited by 45
- 2020Unsupervised Multi-hop Question Answering by Question Generationcited by 38
- 2019Span Selection Pre-training for Question Answeringcited by 36
- 2020ConVEx: Data-Efficient and Few-Shot Slot Labelingcited by 22
- 2021ReasonBERT: Pre-trained to Reason with Distant Supervisioncited by 22
- 2021Database Reasoning Over Textcited by 20
- 2022STable: Table Generation Framework for Encoder-Decoder Modelscited by 15
- 2021GenSF: Simultaneous Adaptation of Generative Pre-trained Models and Slot Fillingcited by 11
- 2019Subjective Databasescited by 7
Built on by
Citing papers per year: 2023: 1.
1 paper.