mlsanityData to 7 Oct 2026

Enhancing Assessment of Self-Consistency in LLM Explanations using Perturbation Strength

arXiv 2609.30849 ↗First posted 25 Sep 2026cs.CLcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation also fits: Test-time compute and efficient thinking

Abstract (arXiv)

Prior work has examined the self-consistency of LLM-generated explanations using surface-level perturbation methods. However, the strength of these perturbations is not explicitly measured and controlled. In this work, we propose an LLM-as-a-judge approach to measure perturbation strength in a unified manner across input and CoT perturbations. We then evaluate the self-consistency in explanations generated from various LLMs under controlled strength conditions, ensuring a fair comparison across perturbation types. Experiments show that our proposed LLM-based perturbation strength measure outperforms other embedding- and probability-based approaches and that input perturbations generally affect LLMs more strongly than CoT perturbations. Our work suggests that judgments about a model's self-consistency is fair only within the same perturbation type.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 9 of its 39 references that are in the map. First posted on arXiv 25 Sep 2026.

Builds on

9 of 39 references.

References · 9

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  3. 2023Mistral 7Bcited by 3,065
  4. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  5. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
  6. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 668
  7. 2020On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behaviorcited by 115
  8. 2025A Survey on Data Contamination for Large Language Modelscited by 33
  9. 2025Noiser: Bounded Input Perturbations for Attributing Large Language Modelscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023On Measuring Faithfulness or Self-consistency of Natural Language Explanations
  2. 2025Bridging Internal Probability and Self-Consistency for Effective and Efficient LLM Reasoning
  3. 2026ICE: Intervention-Consistent Explanation Evaluation with Statistical Grounding for LLMs
  4. 2025A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning
  5. 2024Lachesis: Predicting LLM Inference Accuracy using Structural Properties of Reasoning Paths
  6. 2026Self-Consistency from Only Two Samples: CoT-PoT Ensembling for Efficient LLM Reasoning
  7. 2026Do LLM Self-Explanations Help Users Predict Model Behavior? Evaluating Counterfactual Simulatability with Pragmatic Perturbations
  8. 2026Can We Trust LLM's Logic? Quantifying Uncertainty, Coherence, and Robustness via a Graph-Based Framework