mlsanityData to 7 Oct 2026

Project Aletheia: Verifier-Guided Distillation of Backtracking for Small Language Models

arXiv 2601.14290 ↗First posted 14 Jan 2026cs.CLcited by 6

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Small Language Models (SLMs, under 10B parameters) are attractive for private, on-device deployment, yet they frequently fail on strict constraint-satisfaction problems due to linear, overconfident reasoning traces that do not recover from early mistakes. We introduce Verifier-Guided Distillation, a training protocol that transfers the process of error repair - explicit conflict detection and backtracking - rather than only correct final answers. By training a 7B model on verified reasoning traces that include mistakes and self-corrections, we show that latent verification behavior can emerge in small models, enabling them to occasionally stop, detect contradictions, and revise earlier assumptions.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 6 of its 10 references that are in the map. First posted on arXiv 14 Jan 2026.

Builds on

6 of 10 references.

References · 6

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  4. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  5. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  6. 2025SATBench: Benchmarking LLMs' Logical Reasoning via Automated Puzzle Generation from SAT Formulascited by 30

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Multi-Rollout On-Policy Distillation via Peer Successes and Failurescited by 8
  2. 2026LemmaBench: A Live, Research-Level Benchmark to Evaluate LLM Capabilities in Mathematicscited by 7
  3. 2026ProofCouncil: An LLM Agent for Solving Open Mathematical Problemscited by 6
  4. 2026Large Language Model Post-Training: A Unified View of Off-Policy and On-Policy Learningcited by 5
  5. 2026QED-Nano: Teaching a Tiny Model to Prove Hard Theoremscited by 2
  6. 2026Decision Shifts, Lost Label Functionality, and an Inconclusive Grounding Audit in Correctness-Gated Multi-Teacher Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Lightweight Latent Verifiers for Efficient Meta-Generation Strategies
  2. 2024Small Language Models Need Strong Verifiers to Self-Correct Reasoning
  3. 2026Verify to Amplify: Improving Reasoning via Learned Chain-of-Thought Verification
  4. 2026VibeThinker-3B: Exploring the Frontier of Verifiable Reasoning in Small Language Models
  5. 2024Learning to Check: Unleashing Potentials for Self-Correction in Large Language Models
  6. 2026Can Language Models Learn to Reject Their Own Bad Reasoning Steps?
  7. 2026ReTreVal: Reasoning Tree with Validation and Cross-Problem Memory for Large Language Models
  8. 2025VerifiAgent: a Unified Verification Agent in Language Model Reasoning