mlsanityData to 14 Oct 2026

Dipper: Diversity in Prompts for Producing Large Language Model Ensembles in Reasoning tasks

arXiv 2412.15238 ↗First posted 12 Dec 2024cs.CL cs.AI cs.LGcited by 22

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large Language Models (LLMs), particularly smaller variants, still struggle with complex reasoning tasks. While inference-time prompting can guide reasoning, existing methods often rely on sequential queries. Ensemble approaches offer a promising path to performance gains, especially given recent batch inference speed-ups. This work introduces DIPPER, a novel, training-free framework that transforms a single LLM into an effective inference-time ensemble. By feeding the model an optimized and diverse set of prompts in parallel, DIPPER elicits varied reasoning paths, leading to performance gains. We empirically demonstrate significant improvements on reasoning benchmarks, such as MATH, where a DIPPER ensemble of three Qwen2-MATH-1.5B instances (via parallel prompting of a single model) outperforms a larger 7B model.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 24 of its 38 references that are in the map. First posted on arXiv 12 Dec 2024.

Builds on

24 of 38 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debate23 May
  2. 2022Towards Reasoning in Large Language Models: A Survey20 Dec
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Models6 Oct

Other references · 21

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  7. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  8. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  9. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  10. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  11. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 598
  12. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 564
9 more
  1. 2021Ensemble deep learning: A reviewcited by 284
  2. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 268
  3. 2022Reasoning with Language Model Prompting: A Surveycited by 252
  4. 2023Progressive-Hint Prompting Improves Reasoning in Large Language Modelscited by 147
  5. 2023Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselvescited by 123
  6. 2024Ensemble Learning for Heterogeneous Large Language Models with Deep Parallel Collaborationcited by 67
  7. 2020Neural Ensemble Search for Uncertainty Estimation and Dataset Shiftcited by 56
  8. 2024Prompt Optimization with EASE? Efficient Ordering-aware Automated Selection of Exemplarscited by 37
  9. 2024Efficient Prompt Caching via Embedding Similaritycited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 11.

22 papers.

Sort:

By area:

  1. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 223
  2. 2025Roll the dice & look before you leap: Going beyond the creative limits of next-token predictioncited by 29
  3. 2025On the Effect of Sampling Diversity in Scaling LLM Inferencecited by 18
  4. 2025LLM-Based Social Simulations Require a Boundarycited by 12
  5. 2026Uncertainty Quantification for Multimodal Large Language Models with Incoherence-adjusted Semantic Volumecited by 8
  6. 2025Multi-Novelty: Improve the Diversity and Novelty of Contents Generated by Large Language Models via inference-time Multi-Views Brainstormingcited by 4
  7. 2025ConlangCrafter: Constructing Languages with a Multi-Hop LLM Pipelinecited by 4
  8. 2026Hear Both Sides: Efficient Multi-Agent Debate via Diversity-Aware Message Retentioncited by 4
  9. 2025Beyond Quality: Unlocking Diversity in Ad Headline Generation with Large Language Modelscited by 3
  10. 2026Within-Model vs Between-Prompt Variability in Large Language Models for Creative Taskscited by 3
  11. 2026Real-Time Trustworthiness Scoring for LLM Structured Outputs and Data Extractioncited by 3
  12. 2026Mixture of Complementary Agents for Robust LLM Ensemblecited by 3
10 more
  1. 2025PIED: Physics-Informed Experimental Design for Inverse Problemscited by 2
  2. 2025Broaden your SCOPE! Efficient Multi-turn Conversation Planning for LLMs with Semantic Spacecited by 2
  3. 2025CIEGAD: Cluster-Conditioned Interpolative and Extrapolative Framework for Geometry-Aware and Domain-Aligned Data Augmentationcited by 1
  4. 2026Forty Shades of Blue: Quality-Diversity Alignment via Mode-Conditioned Reinforcement Learningcited by 1
  5. 2025Uncovering Scaling Laws for Large Language Models via Inverse Problemscited by 0
  6. 2026Small Language Model Helps Resolve Semantic Ambiguity of LLM Promptcited by 0
  7. 2026KACE: Knowledge-Adaptive Context Engineering for Mathematical Reasoningcited by 0
  8. 2026MOSCOPT: Mixture-of-Skills Collective Optimization for LLM Agentscited by 0
  9. 2026Efficient Reasoning Exploration via State-Conditioned Latent Steering with Progress Guidancecited by 0
  10. 2026Two Heads Are Better Than One: Aggregating Weaker LLMs for Better Forecastscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Boosted Prompt Ensembles for Large Language Models
  2. 2023Batch Prompting: Efficient Inference with Large Language Model APIs
  3. 2023Diversity of Thought Improves Reasoning Abilities of LLMs
  4. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches
  5. 2025PromptCoT 2.0: Scaling Prompt Synthesis for Large Language Model Reasoning
  6. 2024Aggregation of Reasoning: A Hierarchical Framework for Enhancing Answer Selection in Large Language Models
  7. 2026Enhancing reasoning accuracy in large language models during inference time
  8. 2024A NotSo Simple Way to Beat Simple Bench