mlsanityData to 7 Oct 2026

Language Models as Critical Thinking Tools: A Case Study of Philosophers

arXiv 2404.04516 ↗First posted 6 Apr 2024cs.HC cs.CL cs.CYcited by 6

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Current work in language models (LMs) helps us speed up or even skip thinking by accelerating and automating cognitive work. But can LMs help us with critical thinking -- thinking in deeper, more reflective ways which challenge assumptions, clarify ideas, and engineer new concepts? We treat philosophy as a case study in critical thinking, and interview 21 professional philosophers about how they engage in critical thinking and on their experiences with LMs. We find that philosophers do not find LMs to be useful because they lack a sense of selfhood (memory, beliefs, consistency) and initiative (curiosity, proactivity). We propose the selfhood-initiative model for critical thinking tools to characterize this gap. Using the model, we formulate three roles LMs could play as critical thinking tools: the Interlocutor, the Monitor, and the Respondent. We hope that our work inspires LM researchers to further develop LMs as critical thinking tools and philosophers and other 'critical thinkers' to imagine intellectually substantive uses of LMs.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 43 of its 156 references that are in the map. First posted on arXiv 6 Apr 2024.

Builds on

43 of 156 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Code Llama: Open Foundation Models for Code24 Aug

Other references · 42

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  4. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  5. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  6. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 752
  7. 2020Aligning AI With Shared Human Valuescited by 706
  8. 2020SummEval: Re-evaluating Summarization Evaluationcited by 666
  9. 2023Evaluating Large Language Models in Theory of Mind Taskscited by 217
  10. 2023Evaluating the Moral Beliefs Encoded in LLMscited by 211
  11. 2023Can large language models provide useful feedback on research papers? A large-scale empirical analysiscited by 195
  12. 2022Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionalscited by 191
30 more
  1. 2023Augmenting Language Models with Long-Term Memorycited by 186
  2. 2022Language Models as Agent Modelscited by 131
  3. 2023Pearl: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrieverscited by 91
  4. 2021Sparks: Inspiration for Science Writing using Language Modelscited by 80
  5. 2023RewriteLM: An Instruction-Tuned Large Language Model for Text Rewritingcited by 67
  6. 2023NormBank: A Knowledge Bank of Situational Social Normscited by 60
  7. 2021MERMAID: Metaphor Generation with Symbolism and Discriminative Decodingcited by 49
  8. 2022Beyond Text Generation: Supporting Writers with Continuous Automatic Text Summariescited by 47
  9. 2022Read, Revise, Repeat: A System Demonstration for Human-in-the-loop Iterative Text Revisioncited by 46
  10. 2024AI-Augmented Brainwriting: Investigating the use of LLMs in group ideationcited by 43
  11. 20223DALL-E: Integrating Text-to-Image AI in 3D Design Workflowscited by 41
  12. 2023Can large language models replace humans in the systematic review process? Evaluating GPT-4's efficacy in screening and extracting data from peer-reviewed and grey literature in multiple languagescited by 36
  13. 2024Improving the Robustness of Large Language Models via Consistency Alignmentcited by 36
  14. 2023CoQuest: Exploring Research Question Co-Creation with an LLM-based Agentcited by 34
  15. 2023Creativity Support in the Age of Large Language Models: An Empirical Study Involving Emerging Writerscited by 32
  16. 2022Scim: Intelligent Skimming Support for Scientific Paperscited by 27
  17. 2022LaMPost: Design and Evaluation of an AI-assisted Email Writing Prototype for Adults with Dyslexiacited by 27
  18. 2023Balancing Autonomy and Alignment: A Multi-Dimensional Taxonomy for Autonomous LLM-powered Multi-Agent Architecturescited by 25
  19. 2024Generating Automatic Feedback on UI Mockups with Large Language Modelscited by 23
  20. 2023Scientists' Perspectives on the Potential for Generative AI in their Fieldscited by 22
  21. 2023Creating a Large Language Model of a Philosophercited by 20
  22. 2021CharacterChat: Supporting the Creation of Fictional Characters through Conversation and Progressive Manifestation with a Chatbotcited by 18
  23. 2023Unveiling Theory of Mind in Large Language Models: A Parallel to Single Neurons in the Human Braincited by 18
  24. 2024Rambler: Supporting Writing With Speech via LLM-Assisted Gist Manipulationcited by 18
  25. 2023Conceptual Design Generation Using Large Language Modelscited by 13
  26. 2024Eliciting Personality Traits in Large Language Modelscited by 12
  27. 2024What if LLMs Have Different World Views: Simulating Alien Civilizations with LLM-based Agentscited by 12
  28. 2023LLMs as Potential Brainstorming Partners for Math and Science Problemscited by 6
  29. 2024Towards Full Authorship with AI: Supporting Revision with AI-Generated Viewscited by 4
  30. 2024OpineBot: Class Feedback Reimagined Using a Conversational LLMcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 3.

6 papers.

  1. 2025Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providerscited by 67
  2. 2025Levels of Autonomy for AI Agentscited by 45
  3. 2024ProgressGym: Alignment with a Millennium of Moral Progresscited by 9
  4. 2026Architecting Trust in Artificial Epistemic Agentscited by 3
  5. 2026Generative Confidants: How do People Experience Trust in Emotional Support from Generative AI?cited by 0
  6. 2026Promoting Critical Thinking With Domain-Specific Generative AI Provocationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025THiNK: Can Large Language Models Think-aloud?
  2. 2024A Philosophical Introduction to Language Models -- Part I: Continuity With Classic Debates
  3. 2025Beyond Pattern Recognition: Probing Mental Representations of LMs
  4. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate
  5. 2025Why LLMs Cannot Think and How to Fix It
  6. 2025Toward Reasonable Parrots: Why Large Language Models Should Argue with Us by Design
  7. 2025On the Thinking-Language Modeling Gap in Large Language Models
  8. 2025Reasoning LLMs are Wandering Solution Explorers