# TIAR: Trajectory-Informed Advantage Reweighting for LLM Abstention Learning

arXiv: https://arxiv.org/abs/2605.25850 · HTML: https://mlsanity.com/paper/2605.25850/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 26 references that are in the map.
- First posted on arXiv 25 May 2026.

## Builds on (most cited first, 14 of 14)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [LLM Self Defense: By Self Examination, LLMs Know They Are Being Tricked](https://mlsanity.com/paper/2308.07308/) (2023)
- [AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions](https://mlsanity.com/paper/2506.09038/) (2025)
- [I Don't Know: Explicit Modeling of Uncertainty with an [IDK] Token](https://mlsanity.com/paper/2412.06676/) (2024)
- [Mitigating LLM Hallucination via Behaviorally Calibrated Reinforcement Learning](https://mlsanity.com/paper/2512.19920/) (2025)
- [Mitigating Hallucination in Abstractive Summarization with Domain-Conditional Mutual Information](https://mlsanity.com/paper/2404.09480/) (2024)
- [QuantLRM: Quantization of Large Reasoning Models via Fine-Tuning Signals](https://mlsanity.com/paper/2602.02581/) (2026)
- [Fine-Tuned Large Language Models for Logical Translation: Reducing Hallucinations with Lang2Logic](https://mlsanity.com/paper/2512.02987/) (2025)
- [LangSAT: A Novel Framework Combining NLP and Reinforcement Learning for SAT Solving](https://mlsanity.com/paper/2512.04374/) (2025)

## Built on by (most cited first, 3 of 3)

- [Uncertainty Quantification for LLM Agents: A Taxonomy, an Evaluation Protocol, and an Empirical Study](https://mlsanity.com/paper/2609.07395/) (2026)
- [Strong yet Controllable LLM Abstention without Learning to Abstain](https://mlsanity.com/paper/2610.16318/) (2026)
- [Training Language Models to Fail Gracefully](https://mlsanity.com/paper/2610.20401/) (2026)

## Abstract (arXiv)

This paper investigates large language model (LLM) abstention learning, specifically using ternary reward, which incentivize truthfulness in large language models. This paper extends that idea by moving from a ternary reward to a Trajectory-Informed advantage reweighting, dynamically re-weights the abstention reward during Group Relative Policy Optimization (GRPO) training. The objective of this work focuses on abstention learning instead of improving truthfulness, serving as an exploration into hallucination reduction. The novelty of this paper lies in methodological innovation, advantage re-weighting, and benchmark selection. Leveraging GRPO's multiple trajectories as a natural abstention signal, this method uses a reward signal to explore knowledge boundaries and encourage consistency. By demonstrating that trajectories can be used as a confidence indicator of the policy relative to the query, they are then used to dynamically calculate the abstention advantage. AbstentionBench is used as the evaluation benchmark, as this work aims to contribute to the field of abstention learning. All datasets on the benchmark were tested against this method and various baselines. Empirical results demonstrate that TIAR achieves state-of-the-art abstention F1 scores across five of six evaluation categories, outperforming the static ternary baseline on 17 of 31 benchmark datasets while fully preserving baseline accuracy.