mlsanityData to 7 Oct 2026

Conceptual Engineering Using Large Language Models

arXiv 2312.03749 ↗First posted 1 Dec 2023cs.CL cs.AI cs.CYcited by 0

Not on the map. Why?

Abstract (arXiv)

We describe a method, based on Jennifer Nado's proposal for classification procedures as targets of conceptual engineering, that implements such procedures by prompting a large language model. We apply this method, using data from the Wikidata knowledge graph, to evaluate stipulative definitions related to two paradigmatic conceptual engineering projects: the International Astronomical Union's redefinition of PLANET and Haslanger's ameliorative analysis of WOMAN. Our results show that classification procedures built using our approach can exhibit good classification performance and, through the generation of rationales for their classifications, can contribute to the identification of issues in either the definitions or the data against which they are being evaluated. We consider objections to this method, and discuss implications of this work for three aspects of theory and practice of conceptual engineering: the definition of its targets, empirical methods for their investigation, and their practical roles. The data and code used for our experiments, together with the experimental results, are available in a Github repository.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 91 references that are in the map. First posted on arXiv 1 Dec 2023.

Builds on

27 of 91 references.

References · 27

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  5. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  6. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  7. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  8. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  9. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  10. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  11. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  12. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 737
15 more
  1. 2020Knowledge Graphscited by 619
  2. 2022Towards Reasoning in Large Language Models: A Surveycited by 593
  3. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 590
  4. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 506
  5. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 331
  6. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 317
  7. 2022The Unreliability of Explanations in Few-shot Prompting for Textual Reasoningcited by 169
  8. 2023SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoningcited by 157
  9. 2021Few-Shot Self-Rationalization with Natural Language Promptscited by 83
  10. 2023Knowledge Graphs: Opportunities and Challengescited by 70
  11. 2023Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrongcited by 51
  12. 2023Prompting is not a substitute for probability measurements in large language modelscited by 43
  13. 2020Knowledge Graphs on the Web -- an Overviewcited by 24
  14. 2023Toward General Design Principles for Generative AI Applicationscited by 20
  15. 2023Large Language Models as Subpopulation Representative Models: A Reviewcited by 16

Built on by

Not cited yet.