mlsanityData to 14 Oct 2026

ELLIS Alicante at CQs-Gen 2025: Winning the critical thinking questions shared task: LLM-based question generation and selection

arXiv 2506.14371 ↗First posted 17 Jun 2025cs.CL cs.HCcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

The widespread adoption of chat interfaces based on Large Language Models (LLMs) raises concerns about promoting superficial learning and undermining the development of critical thinking skills. Instead of relying on LLMs purely for retrieving factual information, this work explores their potential to foster deeper reasoning by generating critical questions that challenge unsupported or vague claims in debate interventions. This study is part of a shared task of the 12th Workshop on Argument Mining, co-located with ACL 2025, focused on automatic critical question generation. We propose a two-step framework involving two small-scale open source language models: a Questioner that generates multiple candidate questions and a Judge that selects the most relevant ones. Our system ranked first in the shared task competition, demonstrating the potential of the proposed LLM-based approach to encourage critical engagement with argumentative texts.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 13 of its 22 references that are in the map. First posted on arXiv 17 Jun 2025.

Builds on

13 of 22 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Benchmarking Critical Questions Generation: A Challenging Reasoning Task for Large Language Models16 May
  2. 2024Enhancing Critical Thinking in Education by means of a Socratic Chatbot9 Sep

Other references · 11

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2024Qwen2.5 Technical Reportcited by 4,942
  6. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  7. 2025Gemma 3 Technical Reportcited by 1,824
  8. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 494
  9. 2023Diversifying Question Generation over Knowledge Base via External Natural Questionscited by 6
  10. 2025An Explainable Framework for Misinformation Identification via Critical Question Answeringcited by 4
  11. 2024Critical Questions Generation: Motivation and Challengescited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025DayDreamer at CQs-Gen 2025: Generating Critical Questions through Argument Scheme Completion
  2. 2025Generate-Then-Validate: A Novel Question Generation Approach Using Small Language Models
  3. 2024The Future of Learning in the Age of Generative AI: Automated Question Generation and Assessment with Large Language Models
  4. 2025Large Language Models in Argument Mining: A Survey
  5. 2024Critical-Questions-of-Thought: Steering LLM reasoning with Argumentative Querying
  6. 2026ArgBench: Benchmarking LLMs on Computational Argumentation Tasks
  7. 2025DS@GT at Touch'e: Large Language Models for Retrieval-Augmented Debate
  8. 2024Leveraging Large Language Models to Generate Course-specific Semantically Annotated Learning Objects