# Question Answering over Electronic Devices: A New Benchmark Dataset and a Multi-Task Learning based QA Framework

arXiv: https://arxiv.org/abs/2109.05897 · HTML: https://mlsanity.com/paper/2109.05897/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 41 references that are in the map.
- First posted on arXiv 13 Sep 2021.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [Latent Retrieval for Weakly Supervised Open Domain Question Answering](https://mlsanity.com/paper/1906.00300/) (2019)
- [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://mlsanity.com/paper/1705.02364/) (2017)
- [FinBERT: Financial Sentiment Analysis with Pre-trained Language Models](https://mlsanity.com/paper/1908.10063/) (2019)
- [Document Expansion by Query Prediction](https://mlsanity.com/paper/1904.08375/) (2019)
- [ERNIE 2.0: A Continual Pre-training Framework for Language Understanding](https://mlsanity.com/paper/1907.12412/) (2019)
- [Episodic Memory in Lifelong Language Learning](https://mlsanity.com/paper/1906.01076/) (2019)
- [Neural Unsupervised Domain Adaptation in NLP---A Survey](https://mlsanity.com/paper/2006.00632/) (2020)
- [LEGAL-BERT: The Muppets straight out of Law School](https://mlsanity.com/paper/2010.02559/) (2020)
- [One Epoch Is All You Need](https://mlsanity.com/paper/1906.06669/) (2019)
- [The TechQA Dataset](https://mlsanity.com/paper/1911.02984/) (2019)
- [A Simple and Effective Model for Answering Multi-span Questions](https://mlsanity.com/paper/1909.13375/) (2019)
- [Retrieve-and-Read: Multi-task Learning of Information Retrieval and Reading Comprehension](https://mlsanity.com/paper/1808.10628/) (2018)
- [An Empirical Investigation Towards Efficient Multi-Domain Language Model Pre-training](https://mlsanity.com/paper/2010.00784/) (2020)

## Built on by (most cited first, 21 of 21)

- [RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2407.11005/) (2024)
- [GFM-RAG: Graph Foundation Model for Retrieval Augmented Generation](https://mlsanity.com/paper/2502.01113/) (2025)
- [CARROT: A Cost Aware Rate Optimal Router](https://mlsanity.com/paper/2502.03261/) (2025)
- [Product Question Answering in E-Commerce: A Survey](https://mlsanity.com/paper/2302.08092/) (2023)
- [MPMQA: Multimodal Question Answering on Product Manuals](https://mlsanity.com/paper/2304.09660/) (2023)
- [EnronQA: Towards Personalized RAG over Private Documents](https://mlsanity.com/paper/2505.00263/) (2025)
- [CheckManual: A New Challenge and Benchmark for Manual-based Appliance Manipulation](https://mlsanity.com/paper/2506.09343/) (2025)
- [Corpus2Skill: Distilling Enterprise Knowledge into Navigable Agent Skills for QA and RAG](https://mlsanity.com/paper/2604.14572/) (2026)
- [Luna: An Evaluation Foundation Model to Catch Language Model Hallucinations with High Accuracy and Low Cost](https://mlsanity.com/paper/2406.00975/) (2024)
- [PAGED: A Benchmark for Procedural Graphs Extraction from Documents](https://mlsanity.com/paper/2408.03630/) (2024)
- [Order-Based Pre-training Strategies for Procedural Text Understanding](https://mlsanity.com/paper/2404.04676/) (2024)
- [Winning Big with Small Models: Knowledge Distillation vs. Self-Training for Reducing Hallucination in Product QA Agents](https://mlsanity.com/paper/2502.19545/) (2025)
- [Data-efficient Meta-models for Evaluation of Context-based Questions and Answers in LLMs](https://mlsanity.com/paper/2505.23299/) (2025)
- [Retrieving Minimal and Sufficient Reasoning Subgraphs with Graph Foundation Models for Path-aware GraphRAG](https://mlsanity.com/paper/2603.07179/) (2026)
- [PrismRAG: Boosting RAG Factuality with Distractor Resilience and Strategized Reasoning](https://mlsanity.com/paper/2507.18857/) (2025)
- [Understanding the Behaviors of Environment-aware Information Retrieval](https://mlsanity.com/paper/2606.16817/) (2026)
- [FastDoc: Domain-Specific Fast Continual Pre-training Technique using Document-Level Metadata and Taxonomy](https://mlsanity.com/paper/2306.06190/) (2023)
- [CLMSM: A Multi-Task Learning Framework for Pre-training on Procedural Text](https://mlsanity.com/paper/2310.14326/) (2023)
- [Sparse Graph Representations for Procedural Instructional Documents](https://mlsanity.com/paper/2402.03957/) (2024)
- [Answering Students' Questions on Course Forums Using Multiple Chain-of-Thought Reasoning and Finetuning RAG-Enabled LLM](https://mlsanity.com/paper/2511.09831/) (2025)
- [GLM-RAG: Graph Language Models for Graph-Based Retrieval-Augmented Generation](https://mlsanity.com/paper/2607.28397/) (2026)

## Abstract (arXiv)

Answering questions asked from instructional corpora such as E-manuals, recipe books, etc., has been far less studied than open-domain factoid context-based question answering. This can be primarily attributed to the absence of standard benchmark datasets. In this paper we meticulously create a large amount of data connected with E-manuals and develop suitable algorithm to exploit it. We collect E-Manual Corpus, a huge corpus of 307,957 E-manuals and pretrain RoBERTa on this large corpus. We create various benchmark QA datasets which include question answer pairs curated by experts based upon two E-manuals, real user questions from Community Question Answering Forum pertaining to E-manuals etc. We introduce EMQAP (E-Manual Question Answering Pipeline) that answers questions pertaining to electronics devices. Built upon the pretrained RoBERTa, it harbors a supervised multi-task learning framework which efficiently performs the dual tasks of identifying the section in the E-manual where the answer can be found and the exact answer span within that section. For E-Manual annotated question-answer pairs, we show an improvement of about 40% in ROUGE-L F1 scores over the most competitive baseline. We perform a detailed ablation study and establish the versatility of EMQAP across different circumstances. The code and datasets are shared at https://github.com/abhi1nandy2/EMNLP-2021-Findings, and the corresponding project website is https://sites.google.com/view/emanualqa/home.