# DeFine: Decision-Making with Analogical Reasoning over Factor Profiles

arXiv: https://arxiv.org/abs/2410.01772 · HTML: https://mlsanity.com/paper/2410.01772/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 38 of its 66 references that are in the map.
- First posted on arXiv 2 Oct 2024.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 30 of 38)

- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Towards Reasoning in Large Language Models: A Survey](https://mlsanity.com/paper/2212.10403/) (2022)
- [FinQA: A Dataset of Numerical Reasoning over Financial Data](https://mlsanity.com/paper/2109.00122/) (2021)
- [TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance](https://mlsanity.com/paper/2105.07624/) (2021)
- [Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models](https://mlsanity.com/paper/2305.19187/) (2023)
- [Principled Reinforcement Learning with Human Feedback from Pairwise or K-wise Comparisons](https://mlsanity.com/paper/2301.11270/) (2023)
- [Emergent Analogical Reasoning in Large Language Models](https://mlsanity.com/paper/2212.09196/) (2022)
- [MuSR: Testing the Limits of Chain-of-thought with Multistep Soft Reasoning](https://mlsanity.com/paper/2310.16049/) (2023)
- [Large Language Models as Analogical Reasoners](https://mlsanity.com/paper/2310.01714/) (2023)
- [Approaching Human-Level Forecasting with Language Models](https://mlsanity.com/paper/2402.18563/) (2024)
- [A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges](https://mlsanity.com/paper/2406.11903/) (2024)
- [Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systems](https://mlsanity.com/paper/2407.01370/) (2024)
- [Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey](https://mlsanity.com/paper/2404.01869/) (2024)
- [DocFinQA: A Long-Context Financial Reasoning Dataset](https://mlsanity.com/paper/2401.06915/) (2024)
- [Linguistic Calibration of Long-Form Generations](https://mlsanity.com/paper/2404.00474/) (2024)
- [ECTSum: A New Benchmark Dataset For Bullet Point Summarization of Long Earnings Call Transcripts](https://mlsanity.com/paper/2210.12467/) (2022)
- [Is In-Context Learning in Large Language Models Bayesian? A Martingale Perspective](https://mlsanity.com/paper/2406.00793/) (2024)
- [Determinants of LLM-assisted Decision-Making](https://mlsanity.com/paper/2402.17385/) (2024)
- [LLM Voting: Human Choices and AI Collective Decision Making](https://mlsanity.com/paper/2402.01766/) (2024)
- [Using Counterfactual Tasks to Evaluate the Generality of Analogical Reasoning in Large Language Models](https://mlsanity.com/paper/2402.08955/) (2024)
- [Thought Propagation: An Analogical Approach to Complex Reasoning with Large Language Models](https://mlsanity.com/paper/2310.03965/) (2023)
- [Evaluating LLMs' Mathematical Reasoning in Financial Document Question Answering](https://mlsanity.com/paper/2402.11194/) (2024)
- [ThinkSum: Probabilistic reasoning over sets using large language models](https://mlsanity.com/paper/2210.01293/) (2022)
- [DeLLMa: Decision Making Under Uncertainty with Large Language Models](https://mlsanity.com/paper/2402.02392/) (2024)
- [BIRD: A Trustworthy Bayesian Inference Framework for Large Language Models](https://mlsanity.com/paper/2404.12494/) (2024)
- [ANALOGYKB: Unlocking Analogical Reasoning of Language Models with A Million-scale Knowledge Base](https://mlsanity.com/paper/2305.05994/) (2023)
- [DecipherPref: Analyzing Influential Factors in Human Preference Judgments via GPT-4](https://mlsanity.com/paper/2305.14702/) (2023)
- [LLMs as Models for Analogical Reasoning](https://mlsanity.com/paper/2406.13803/) (2024)
- [An Exploratory Study of Stock Price Movements from Earnings Calls](https://mlsanity.com/paper/2203.12460/) (2022)
- [Response: Emergent analogical reasoning in large language models](https://mlsanity.com/paper/2308.16118/) (2023)

## Built on by (most cited first, 4 of 4)

- [Code to Think, Think to Code: A Survey on Code-Enhanced Reasoning and Reasoning-Driven Code Intelligence in LLMs](https://mlsanity.com/paper/2502.19411/) (2025)
- [Rethinking Prospect Theory for LLMs: Revealing the Instability of Decision-Making under Epistemic Uncertainty](https://mlsanity.com/paper/2508.08992/) (2025)
- [CONDESION-BENCH: Conditional Decision-Making of Large Language Models in Compositional Action Space](https://mlsanity.com/paper/2604.09029/) (2026)
- [Representation-Aligned Auxiliary Supervision for Language Model Adaptation](https://mlsanity.com/paper/2610.04098/) (2026)

## Abstract (arXiv)

LLMs are ideal for decision-making thanks to their ability to reason over long contexts. However, challenges arise when processing speech transcripts that describe complex scenarios, as they are verbose and include repetition, hedging, and vagueness. E.g., during a company's earnings call, an executive might project a positive revenue outlook to reassure investors, despite uncertainty regarding future earnings. It is crucial for LLMs to incorporate this uncertainty systematically when making decisions. In this paper, we introduce textscDeFine, a modular framework that constructs probabilistic factor profiles from complex scenarios. It then integrates these profiles with analogical reasoning, leveraging insights from similar past experiences to guide LLMs in making critical decisions in new situations. Our framework separates the tasks of quantifying uncertainty and incorporating it into LLM decision-making. This approach is particularly useful in areas such as consulting and financial deliberation, where making decisions under uncertainty is vital.