MAC-Tuning: LLM Multi-Compositional Problem Reasoning with Enhanced Knowledge Boundary Awareness
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
The hallucination of non-existent facts by LLMs is an important problem given its widespread adoption across various applications. Previous research addresses this problem by analyzing the internal parameterized knowledge boundaries to estimate confidence. However, these studies focus on the single-problem setting and have not explored the more challenging multi-problem setting, which requires accurately answering multiple questions simultaneously. We introduce a novel method for the multi-problem setting, Multiple Answers and Confidence Stepwise Tuning (MAC-Tuning), that separates the learning of answer prediction and confidence estimation during fine-tuning on instruction data. Extensive experiments demonstrate that our method outperforms baselines by up to 25% in average precision.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 19 of its 25 references that are in the map. First posted on arXiv 30 Apr 2025.
Builds on
19 of 25 references.
References · 19
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
- 2018CoQA: A Conversational Question Answering Challengecited by 674
- 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 361
- 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 351
- 2023Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervisioncited by 318
- 2023Batch Prompting: Efficient Inference with Large Language Model APIscited by 84
- 2022A Close Look into the Calibration of Pre-trained Language Modelscited by 57
- 2023Won't Get Fooled Again: Answering Questions with False Premisescited by 47
7 more
- 2025Scaling Laws of Synthetic Data for Language Modelscited by 44
- 2024Learning to Trust Your Feelings: Leveraging Self-awareness in LLMs for Hallucination Mitigationcited by 43
- 2024Teaching Large Language Models to Express Knowledge Boundary from Their Own Signalscited by 32
- 2024Multi-Task Inference: Can Large Language Models Follow Multiple Instructions at Once?cited by 28
- 2023TOME: A Two-stage Approach for Model-based Retrievalcited by 21
- 2023BatchPrompt: Accomplish more with lesscited by 16
- 2024L^2GC:Lorentzian Linear Graph Convolutional Networks for Node Classificationcited by 5
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Diagnosing Multi-step Reasoning Failures in Black-box LLMs via Stepwise Confidence Attribution
- 2024Improving Complex Reasoning over Knowledge Graph with Logic-Aware Curriculum Tuning
- 2026Distilling Reasoning Without Knowledge: A Framework for Reliable LLMs
- 2025Climbing the Ladder of Reasoning: What LLMs Can-and Still Can't-Solve after SFT?
- 2026Mi:dm K 2.5 Pro
- 2025The Thinking Spectrum: An Empirical Study of Tunable Reasoning in LLMs through Model Merging
- 2026The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modes
- 2024KCIF: Knowledge-Conditioned Instruction Following