mlsanityData to 14 Oct 2026

DaVinci at SemEval-2024 Task 9: Few-shot prompting GPT-3.5 for Unconventional Reasoning

arXiv 2405.11559 ↗First posted 19 May 2024cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

While significant work has been done in the field of NLP on vertical thinking, which involves primarily logical thinking, little work has been done towards lateral thinking, which involves looking at problems from an unconventional perspective and defying existing conceptions and notions. Towards this direction, SemEval 2024 introduces the task of BRAINTEASER, which involves two types of questions -- Sentence Puzzles and Word Puzzles that defy conventional common-sense reasoning and constraints. In this paper, we tackle both types of questions using few-shot prompting on GPT-3.5 and gain insights regarding the difference in the nature of the two types. Our prompting strategy placed us 26th on the leaderboard for the Sentence Puzzle and 15th on the Word Puzzle task.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 21 references that are in the map. First posted on arXiv 19 May 2024.

Builds on

13 of 21 references.

References · 13

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  5. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  6. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  7. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  8. 2023A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Modelscited by 246
  9. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matterscited by 222
  10. 2022Leveraging Large Language Models for Multiple Choice Question Answeringcited by 203
  11. 2023Evaluating ChatGPT as a Question Answering System: A Comprehensive Analysis and Comparison with Existing Modelscited by 11
  12. 2023Abstractive Summarization of Large Document Collections Using GPTcited by 6
1 more
  1. 2024SemEval-2024 Task 9: BRAINTEASER: A Novel Task Defying Common Sensecited by 6

Built on by

Not cited yet.