RAGthoven at SemEval-2026 Task 1: A Multi-Stage Pipeline Walks Into a Benchmark and Barely Clears the Bar
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
We present RAGthoven, our system for SemEval-2026 Task 1 (MWAHAHA), Subtask A (multilingual constrained humor generation in English, Spanish, and Chinese). RAGthoven decomposes creative text generation into a multi-stage large language model (LLM) pipeline (Planner, Best-of-N Writer, Reflector for self-critique, LLM-as-a-judge Judge) grounded in computational humor theory (Benign Violation Theory, Script-based Semantic Theory of Humor) and refined across ten experiments. In our final configuration, we augment the Planner with retrieval-augmented generation (RAG) from a curated joke corpus, seeding generation with diverse joke mechanisms. We also evaluate two agentic variants -- ReAct-style sequential tool-calling (Exp09) and autonomous multi-branch orchestration (Exp10) -- that expose the same four stages with a deterministic ConstraintAudit checker. Across four frontier models on a held-out 12-instance English sample, neither agentic variant produced outputs we judged superior to the non-agentic pipeline despite substantially higher tool-call budgets. RAGthoven shares Rank 1 with the Gemini 2.5 Flash baseline in all three languages, with overlapping organizer-reported confidence intervals. In Spanish, it leads the baseline by 42 raw Elo points (1182 vs. 1140), while in English (1045 vs. 1081) and Chinese (1045 vs. 1053) the baseline holds the higher raw rating within the same statistical tie. Together, these results suggest language-dependent diminishing returns from elaborate multi-stage prompt engineering and agentic scaffolding once a strong frontier model is in the loop.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 15 of its 39 references that are in the map. First posted on arXiv 14 Jul 2026.
Builds on
15 of 39 references.
References · 15
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
- 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
- 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 558
- 2023Art or Artifice? Large Language Models and the False Promise of Creativitycited by 172
- 2022Do Androids Laugh at Electric Sheep? Humor "Understanding" Benchmarks from The New Yorker Caption Contestcited by 98
- 2023Metacognitive Prompting Improves Understanding in Large Language Modelscited by 58
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026HumorRank: A Tournament-Based Leaderboard for Evaluating Humor Generation in Large Language Models
- 2025HumorPlanSearch: Structured Planning and HuCoT for Contextual AI Humor
- 2026HumorGen: Cognitive Synergy for Humor Generation in Large Language Models via Persona-Based Distillation
- 2026On the Wings of Imagination: Conflicting Script-based Multi-role Framework for Humor Caption Generation
- 2026Jokeasy: Exploring Human-AI Collaboration in Thematic Joke Generation ↗
- 2025Who's Laughing Now? An Overview of Computational Humour Generation and Explanation
- 2024Small But Funny: A Feedback-Driven Approach to Humor Distillation
- 2026Grounded Satirical Generation with RAG