CAREF: Calibration-Aware Regularization for Explanation Faithfulness Without Rationale Supervision
Not on the map. Why?
Abstract (arXiv)
We introduce CAREF, a parameter-efficient fine-tuning framework that jointly optimizes predictive accuracy and explanation faithfulness via calibration-aware regularization. At its core, CAREF couples entropy-based calibration with token-level sparsity control through a single unified loss, the Calibration-Aware Regularization for Explanation Faithfulness (LSCED), without requiring rationale supervision. Evaluated on four NLE benchmarks (COS-E, ECQA, ComVE, e-SNLI) with Flan-T5, our lightweight CAREF-AQ variant attains the best average accuracy (89.04) and explanation alignment (81.00 nBERT) using only 6.43% of trainable parameters, outperforming LoRA and AdaLoRA. To our knowledge, CAREF is the first method to unify entropy and sparsity regularization in a single training objective for interpretable LLM fine-tuning.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 13 references that are in the map. First posted on arXiv 27 May 2026.
Builds on
8 of 13 references.
References · 8
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
- 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 896
- 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
- 2021Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processingcited by 126
- 2019Does It Make Sense? And Why? A Pilot Study for Sense Making and Explanationcited by 43
- 2020The Struggles of Feature-Based Explanations: Shapley Values vs. Minimal Sufficient Subsetscited by 14
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Entropy Regularization: A Free Correction to Cross-Entropy for Verified Demonstrations
- 2026InstructDiff: Domain-Adaptive Data Selection via Differential Entropy for Efficient LLM Fine-Tuning
- 2025TECP: Token-Entropy Conformal Prediction for LLMs
- 2026Astro: Activation-guided Structured Regularization for Outlier-Robust LLM Post-Training Quantization
- 2025Preserving LLM Capabilities through Calibration Data Curation: From Analysis to Optimization
- 2025EDIT: Early Diffusion Inference Termination for dLLMs Based on Dynamics of Training Gradients
- 2024PACE: Marrying generalization in PArameter-efficient fine-tuning with Consistency rEgularization
- 2025Quaff: Quantized Parameter-Efficient Fine-Tuning under Outlier Spatial Stability Hypothesis