# Does It Make Sense? And Why? A Pilot Study for Sense Making and Explanation

arXiv: https://arxiv.org/abs/1906.00363 · HTML: https://mlsanity.com/paper/1906.00363/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jun 2019.

## Built on by (most cited first, 30 of 43)

- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization](https://mlsanity.com/paper/2306.05087/) (2023)
- [Knowledge Conflicts for LLMs: A Survey](https://mlsanity.com/paper/2403.08319/) (2024)
- [OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization](https://mlsanity.com/paper/2212.12017/) (2022)
- [Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Future](https://mlsanity.com/paper/2309.15402/) (2023)
- [Unified Demonstration Retriever for In-Context Learning](https://mlsanity.com/paper/2305.04320/) (2023)
- [Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processing](https://mlsanity.com/paper/2102.12060/) (2021)
- [The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning](https://mlsanity.com/paper/2305.14045/) (2023)
- [Few-Shot Self-Rationalization with Natural Language Prompts](https://mlsanity.com/paper/2111.08284/) (2021)
- [When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data](https://mlsanity.com/paper/2102.02201/) (2021)
- [Can LLMs Learn from Previous Mistakes? Investigating LLMs' Errors to Boost for Reasoning](https://mlsanity.com/paper/2403.20046/) (2024)
- [Can Generative Pre-trained Language Models Serve as Knowledge Bases for Closed-book QA?](https://mlsanity.com/paper/2106.01561/) (2021)
- [NormBank: A Knowledge Bank of Situational Social Norms](https://mlsanity.com/paper/2305.17008/) (2023)
- [ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoning](https://mlsanity.com/paper/2104.07644/) (2021)
- [Give Us the Facts: Enhancing Large Language Models with Knowledge Graphs for Fact-aware Language Modeling](https://mlsanity.com/paper/2306.11489/) (2023)
- [Going Beyond XAI: A Systematic Survey for Explanation-Guided Learning](https://mlsanity.com/paper/2212.03954/) (2022)
- [Local Interpretations for Explainable Natural Language Processing: A Survey](https://mlsanity.com/paper/2103.11072/) (2021)
- [Assessing Hidden Risks of LLMs: An Empirical Study on Robustness, Consistency, and Credibility](https://mlsanity.com/paper/2305.10235/) (2023)
- [Explanation-based Finetuning Makes Models More Robust to Spurious Cues](https://mlsanity.com/paper/2305.04990/) (2023)
- [Knowledge-Grounded Self-Rationalization via Extractive and Natural Language Explanations](https://mlsanity.com/paper/2106.13876/) (2021)
- [Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Experts](https://mlsanity.com/paper/2203.07285/) (2022)
- [Can Language Models Understand Physical Concepts?](https://mlsanity.com/paper/2305.14057/) (2023)
- [Exploring Generalization Ability of Pretrained Language Models on Arithmetic and Logical Reasoning](https://mlsanity.com/paper/2108.06743/) (2021)
- [How Conservative are Language Models? Adapting to the Introduction of Gender-Neutral Pronouns](https://mlsanity.com/paper/2204.10281/) (2022)
- [Abductive Commonsense Reasoning Exploiting Mutually Exclusive Explanations](https://mlsanity.com/paper/2305.14618/) (2023)
- [Constrained Text Generation with Global Guidance -- Case Study on CommonGen](https://mlsanity.com/paper/2103.07170/) (2021)
- [Fully Authentic Visual Question Answering Dataset from Online Communities](https://mlsanity.com/paper/2311.15562/) (2023)
- [On Evaluating Explanation Utility for Human-AI Decision Making in NLP](https://mlsanity.com/paper/2407.03545/) (2024)
- [ZARA: Improving Few-Shot Self-Rationalization for Small Language Models](https://mlsanity.com/paper/2305.07355/) (2023)
- [CRoW: Benchmarking Commonsense Reasoning in Real-World Tasks](https://mlsanity.com/paper/2310.15239/) (2023)

## Abstract (arXiv)

Introducing common sense to natural language understanding systems has received increasing research attention. It remains a fundamental question on how to evaluate whether a system has a sense making capability. Existing benchmarks measures commonsense knowledge indirectly and without explanation. In this paper, we release a benchmark to directly test whether a system can differentiate natural language statements that make sense from those that do not make sense. In addition, a system is asked to identify the most crucial reason why a statement does not make sense. We evaluate models trained over large-scale language modeling tasks as well as human performance, showing that there are different challenges for system sense making.