mlsanityData to 14 Oct 2026

DeFine: Decision-Making with Analogical Reasoning over Factor Profiles

arXiv 2410.01772 ↗First posted 2 Oct 2024cs.CL cs.AIcited by 4

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

LLMs are ideal for decision-making thanks to their ability to reason over long contexts. However, challenges arise when processing speech transcripts that describe complex scenarios, as they are verbose and include repetition, hedging, and vagueness. E.g., during a company's earnings call, an executive might project a positive revenue outlook to reassure investors, despite uncertainty regarding future earnings. It is crucial for LLMs to incorporate this uncertainty systematically when making decisions. In this paper, we introduce textscDeFine, a modular framework that constructs probabilistic factor profiles from complex scenarios. It then integrates these profiles with analogical reasoning, leveraging insights from similar past experiences to guide LLMs in making critical decisions in new situations. Our framework separates the tasks of quantifying uncertainty and incorporating it into LLM decision-making. This approach is particularly useful in areas such as consulting and financial deliberation, where making decisions under uncertainty is vital.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 38 of its 66 references that are in the map. First posted on arXiv 2 Oct 2024.

Builds on

38 of 66 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022ThinkSum: Probabilistic reasoning over sets using large language models4 Oct

Other references · 37

  1. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  2. 2022Towards Reasoning in Large Language Models: A Surveycited by 596
  3. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 562
  4. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 408
  5. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 293
  6. 2023Principled Reinforcement Learning with Human Feedback from Pairwise or K-wise Comparisonscited by 247
  7. 2022Emergent Analogical Reasoning in Large Language Modelscited by 240
  8. 2023MuSR: Testing the Limits of Chain-of-thought with Multistep Soft Reasoningcited by 201
  9. 2023Large Language Models as Analogical Reasonerscited by 107
  10. 2024Approaching Human-Level Forecasting with Language Modelscited by 107
  11. 2024A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challengescited by 90
  12. 2024Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systemscited by 87
25 more
  1. 2024Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Surveycited by 83
  2. 2024DocFinQA: A Long-Context Financial Reasoning Datasetcited by 60
  3. 2024Linguistic Calibration of Long-Form Generationscited by 60
  4. 2022ECTSum: A New Benchmark Dataset For Bullet Point Summarization of Long Earnings Call Transcriptscited by 59
  5. 2024Is In-Context Learning in Large Language Models Bayesian? A Martingale Perspectivecited by 57
  6. 2024Determinants of LLM-assisted Decision-Makingcited by 56
  7. 2024LLM Voting: Human Choices and AI Collective Decision Makingcited by 48
  8. 2024Using Counterfactual Tasks to Evaluate the Generality of Analogical Reasoning in Large Language Modelscited by 38
  9. 2023Thought Propagation: An Analogical Approach to Complex Reasoning with Large Language Modelscited by 37
  10. 2024Evaluating LLMs' Mathematical Reasoning in Financial Document Question Answeringcited by 25
  11. 2024DeLLMa: Decision Making Under Uncertainty with Large Language Modelscited by 21
  12. 2024BIRD: A Trustworthy Bayesian Inference Framework for Large Language Modelscited by 18
  13. 2023ANALOGYKB: Unlocking Analogical Reasoning of Language Models with A Million-scale Knowledge Basecited by 16
  14. 2023DecipherPref: Analyzing Influential Factors in Human Preference Judgments via GPT-4cited by 16
  15. 2024LLMs as Models for Analogical Reasoningcited by 16
  16. 2022An Exploratory Study of Stock Price Movements from Earnings Callscited by 14
  17. 2023Response: Emergent analogical reasoning in large language modelscited by 13
  18. 2021Stock price prediction using BERT and GANcited by 11
  19. 2023Rational Decision-Making Agent with Internalized Utility Judgmentcited by 11
  20. 2023ARN: Analogical Reasoning on Narrativescited by 8
  21. 2024Relevant or Random: Can LLMs Truly Perform Analogical Reasoning?cited by 8
  22. 2024When Reasoning Meets Information Aggregation: A Case Study with Sports Narrativescited by 7
  23. 2024RiskLabs: Predicting Financial Risk Using Large Language Model based on Multimodal and Multi-Sources Datacited by 6
  24. 2023Methods for Acquiring and Incorporating Knowledge into Stock Price Prediction: A Surveycited by 3
  25. 2024Instruction-Guided Bullet Point Summarization of Long Financial Earnings Call Transcriptscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Code to Think, Think to Code: A Survey on Code-Enhanced Reasoning and Reasoning-Driven Code Intelligence in LLMscited by 47
  2. 2025Rethinking Prospect Theory for LLMs: Revealing the Instability of Decision-Making under Epistemic Uncertaintycited by 4
  3. 2026CONDESION-BENCH: Conditional Decision-Making of Large Language Models in Compositional Action Spacecited by 0
  4. 2026Representation-Aligned Auxiliary Supervision for Language Model Adaptationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Mind the (DH) Gap! A Contrast in Risky Choices Between Reasoning and Conversational LLMs
  2. 2026IDEA: An Interpretable and Editable Decision-Making Framework for LLMs via Verbal-to-Numeric Calibration
  3. 2025Can LLMs Help Improve Analogical Reasoning For Strategic Decisions? Experimental Evidence from Humans and GPT-4
  4. 2025Risk Profiling and Modulation for LLMs
  5. 2026LLMs Should Not Yet Be Credited with Decision Explanation
  6. 2026From Stochastic Answers to Verifiable Reasoning: Interpretable Decision-Making with LLM-Generated Code
  7. 2025DecisionFlow: Advancing Large Language Model as Principled Decision Maker
  8. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs