# Critical Questions Generation: Motivation and Challenges

arXiv: https://arxiv.org/abs/2410.14335 · HTML: https://mlsanity.com/paper/2410.14335/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 43 references that are in the map.
- First posted on arXiv 18 Oct 2024.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [CONAN -- COunter NArratives through Nichesourcing: a Multilingual Dataset of Responses to Fight Online Hate Speech](https://mlsanity.com/paper/1910.03270/) (2019)
- [Are Large Language Models Really Good Logical Reasoners? A Comprehensive Evaluation and Beyond](https://mlsanity.com/paper/2306.09841/) (2023)
- [BERT for Evidence Retrieval and Claim Verification](https://mlsanity.com/paper/1910.02655/) (2019)
- [Large Language Models Are Not Strong Abstract Reasoners](https://mlsanity.com/paper/2305.19555/) (2023)
- [Exploring the Potential of Large Language Models in Computational Argumentation](https://mlsanity.com/paper/2311.09022/) (2023)
- [Multitask Instruction-based Prompting for Fallacy Recognition](https://mlsanity.com/paper/2301.09992/) (2023)
- [How susceptible are LLMs to Logical Fallacies?](https://mlsanity.com/paper/2308.09853/) (2023)
- [Classifying Argumentative Relations Using Logical Mechanisms and Argumentation Schemes](https://mlsanity.com/paper/2105.07571/) (2021)
- [NLAS-multi: A Multilingual Corpus of Automatically Generated Natural Language Argumentation Schemes](https://mlsanity.com/paper/2402.14458/) (2024)
- [MAFALDA: A Benchmark and Comprehensive Study of Fallacy Detection and Classification](https://mlsanity.com/paper/2311.09761/) (2023)

## Built on by (most cited first, 3 of 3)

- [Incivility and Rigidity: Evaluating the Risks of Fine-Tuning LLMs for Political Argumentation](https://mlsanity.com/paper/2411.16813/) (2024)
- [DayDreamer at CQs-Gen 2025: Generating Critical Questions through Argument Scheme Completion](https://mlsanity.com/paper/2505.15554/) (2025)
- [ELLIS Alicante at CQs-Gen 2025: Winning the critical thinking questions shared task: LLM-based question generation and selection](https://mlsanity.com/paper/2506.14371/) (2025)

## Abstract (arXiv)

The development of Large Language Models (LLMs) has brought impressive performances on mitigation strategies against misinformation, such as counterargument generation. However, LLMs are still seriously hindered by outdated knowledge and by their tendency to generate hallucinated content. In order to circumvent these issues, we propose a new task, namely, Critical Questions Generation, consisting of processing an argumentative text to generate the critical questions (CQs) raised by it. In argumentation theory CQs are tools designed to lay bare the blind spots of an argument by pointing at the information it could be missing. Thus, instead of trying to deploy LLMs to produce knowledgeable and relevant counterarguments, we use them to question arguments, without requiring any external knowledge. Research on CQs Generation using LLMs requires a reference dataset for large scale experimentation. Thus, in this work we investigate two complementary methods to create such a resource: (i) instantiating CQs templates as defined by Walton's argumentation theory and (ii), using LLMs as CQs generators. By doing so, we contribute with a procedure to establish what is a valid CQ and conclude that, while LLMs are reasonable CQ generators, they still have a wide margin for improvement in this task.