mlsanityData to 14 Oct 2026

EXPLORA: Efficient Exemplar Subset Selection for Complex Reasoning

arXiv 2411.03877 ↗First posted 6 Nov 2024cs.LGcited by 5

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Answering reasoning-based complex questions over text and hybrid sources, including tables, is a challenging task. Recent advances in large language models (LLMs) have enabled in-context learning (ICL), allowing LLMs to acquire proficiency in a specific task using only a few demonstration samples (exemplars). A critical challenge in ICL is the selection of optimal exemplars, which can be either task-specific (static) or test-example-specific (dynamic). Static exemplars provide faster inference times and increased robustness across a distribution of test examples. In this paper, we propose an algorithm for static exemplar subset selection for complex reasoning tasks. We introduce EXPLORA, a novel exploration method designed to estimate the parameters of the scoring function, which evaluates exemplar subsets without incorporating confidence information. EXPLORA significantly reduces the number of LLM calls to ~11% of those required by state-of-the-art methods and achieves a substantial performance improvement of 12.24%. We open-source our code and data (https://github.com/kiranpurohit/EXPLORA).

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 28 of its 55 references that are in the map. First posted on arXiv 6 Nov 2024.

Builds on

28 of 55 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Measuring and Narrowing the Compositionality Gap in Language Models7 Oct

Other references · 27

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  4. 2022Emergent Abilities of Large Language Modelscited by 2,512
  5. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  6. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
  7. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 938
  8. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  9. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 562
  10. 2016Solving General Arithmetic Word Problemscited by 360
  11. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 350
  12. 2022Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfercited by 288
15 more
  1. 2022Selective Annotation Makes Language Models Better Few-Shot Learnerscited by 243
  2. 2022DeepCore: A Comprehensive Library for Coreset Selection in Deep Learningcited by 147
  3. 2023Compositional Exemplars for In-context Learningcited by 144
  4. 2022Data Curation Alone Can Stabilize In-context Learningcited by 59
  5. 2020Knowledge-Aware Language Model Pretrainingcited by 52
  6. 2022Explainable Information Retrieval: A Surveycited by 15
  7. 2024Misconfidence-based Demonstration Selection for LLM In-Context Learningcited by 14
  8. 2022Despite "super-human" performance, current LLMs are unsuited for decisions about ethics and safetycited by 13
  9. 2023SEER : A Knapsack approach to Exemplar Selection for In-Context HybridQAcited by 9
  10. 2022A Survey on Table-and-Text HybridQA: Concepts, Methods, Challenges and Future Directionscited by 7
  11. 2023Query Understanding in the Age of Large Language Modelscited by 7
  12. 2023Ethical Considerations and Policy Implications for Large Language Models: Guiding Responsible Development and Deploymentcited by 5
  13. 2023DQ-LoRe: Dual Queries with Low Rank Approximation Re-ranking for In-Context Learningcited by 5
  14. 2024DEXTER: A Benchmark for open-domain Complex Question Answering using LLMscited by 4
  15. 2023In-Context Ability Transfer for Question Decomposition in Complex QAcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025Sample Efficient Demonstration Selection for In-Context Learningcited by 13
  2. 2025Credible Plan-Driven RAG Method for Multi-Hop Question Answeringcited by 12
  3. 2025OptiSeq: Ordering Examples On-The-Fly for In-Context Learningcited by 6
  4. 2026Discovering Implicit Large Language Model Alignment Objectivescited by 0
  5. 2026Data Efficient Sample Selection for In-Context Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Reasoning Graph Enhanced Exemplars Retrieval for In-Context Learning
  2. 2024The Power of Adaptation: Boosting In-Context Learning through Adaptive Prompting
  3. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning
  4. 2024Unlocking Reasoning Potential in Large Langauge Models by Scaling Code-form Planning
  5. 2022Complementary Explanations for Effective In-Context Learning
  6. 2024Aggregation of Reasoning: A Hierarchical Framework for Enhancing Answer Selection in Large Language Models
  7. 2024CLR-Fact: Evaluating the Complex Logical Reasoning Capability of Large Language Models over Factual Knowledge
  8. 2024DCR: Divide-and-Conquer Reasoning for Multi-choice Question Answering with LLMs