# Knowing-how & Knowing-that: A New Task for Machine Comprehension of User Manuals

arXiv: https://arxiv.org/abs/2306.04187 · HTML: https://mlsanity.com/paper/2306.04187/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 49 references that are in the map.
- First posted on arXiv 7 Jun 2023.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Tracking State Changes in Procedural Text: A Challenge Dataset and Models for Process Paragraph Comprehension](https://mlsanity.com/paper/1805.06975/) (2018)
- [Unsupervised Question Answering by Cloze Translation](https://mlsanity.com/paper/1906.04980/) (2019)
- [Simulating Action Dynamics with Neural Process Networks](https://mlsanity.com/paper/1711.05313/) (2017)
- [The Materials Science Procedural Text Corpus: Annotating Materials Synthesis Procedures with Shallow Semantic Structures](https://mlsanity.com/paper/1905.06939/) (2019)
- [Simpler but More Accurate Semantic Dependency Parsing](https://mlsanity.com/paper/1807.01396/) (2018)
- [A Dataset for Tracking Entities in Open Domain Procedural Text](https://mlsanity.com/paper/2011.08092/) (2020)
- [N-LTP: An Open-source Neural Language Technology Platform for Chinese](https://mlsanity.com/paper/2009.11616/) (2020)
- [PERT: Pre-training BERT with Permuted Language Model](https://mlsanity.com/paper/2203.06906/) (2022)
- [Second-Order Semantic Dependency Parsing with End-to-End Neural Networks](https://mlsanity.com/paper/1906.07880/) (2019)
- [Adaptations of ROUGE and BLEU to Better Evaluate Machine Reading Comprehension Task](https://mlsanity.com/paper/1806.03578/) (2018)
- [Establishing Strong Baselines for the New Decade: Sequence Tagging, Syntactic and Semantic Parsing with BERT](https://mlsanity.com/paper/1908.04943/) (2019)
- [Knowledge-Aware Procedural Text Understanding with Multi-Stage Training](https://mlsanity.com/paper/2009.13199/) (2020)
- [Procedural Reading Comprehension with Attribute-Aware Context Flow](https://mlsanity.com/paper/2003.13878/) (2020)
- [Annotating and Extracting Synthesis Process of All-Solid-State Batteries from Scientific Literature](https://mlsanity.com/paper/2002.07339/) (2020)
- [Unsupervised Question Answering via Answer Diversifying](https://mlsanity.com/paper/2208.10813/) (2022)
- [Tracking entities in technical procedures -- a new dataset and baselines](https://mlsanity.com/paper/2104.07378/) (2021)

## Abstract (arXiv)

The machine reading comprehension (MRC) of user manuals has huge potential in customer service. However, current methods have trouble answering complex questions. Therefore, we introduce the Knowing-how & Knowing-that task that requires the model to answer factoid-style, procedure-style, and inconsistent questions about user manuals. We resolve this task by jointly representing the steps and facts in a graph TARA, which supports a unified inference of various questions. Towards a systematical benchmarking study, we design a heuristic method to automatically parse user manuals into TARAs and build an annotated dataset to test the model's ability in answering real-world questions. Empirical results demonstrate that representing user manuals as TARAs is a desired solution for the MRC of user manuals. An in-depth investigation of TARA further sheds light on the issues and broader impacts of future representations of user manuals. We hope our work can move the MRC of user manuals to a more complex and realistic stage.