# Do Neural Networks Preserve Case Structure? Case-Based Decomposition, Interpretation, and Decision Consistency

arXiv: https://arxiv.org/abs/2607.11347 · HTML: https://mlsanity.com/paper/2607.11347/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 38 references that are in the map.
- First posted on arXiv 13 Jul 2026.

## Builds on (most cited first, 25 of 25)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Estimating Training Data Influence by Tracing Gradient Descent](https://mlsanity.com/paper/2002.08484/) (2020)
- [Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models](https://mlsanity.com/paper/2403.19647/) (2024)
- [Convergent Learning: Do different neural networks learn the same representations?](https://mlsanity.com/paper/1511.07543/) (2015)
- [Representer Point Selection for Explaining Deep Neural Networks](https://mlsanity.com/paper/1811.09720/) (2018)
- [Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs](https://mlsanity.com/paper/2309.07311/) (2023)
- [DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Models](https://mlsanity.com/paper/2310.00902/) (2023)
- [How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representations](https://mlsanity.com/paper/2310.10616/) (2023)
- [ProtoPFormer: Concentrating on Prototypical Parts in Vision Transformers for Interpretable Image Recognition](https://mlsanity.com/paper/2208.10431/) (2022)
- [Intriguing Properties of Data Attribution on Diffusion Models](https://mlsanity.com/paper/2311.00500/) (2023)
- [Scalable Influence and Fact Tracing for Large Language Model Pretraining](https://mlsanity.com/paper/2410.17413/) (2024)
- [Capturing the Temporal Dependence of Training Data Influence](https://mlsanity.com/paper/2412.09538/) (2024)
- [Energy-Based Concept Bottleneck Models: Unifying Prediction, Concept Intervention, and Probabilistic Interpretations](https://mlsanity.com/paper/2401.14142/) (2024)
- [Efficient Sketches for Training Data Attribution and Studying the Loss Landscape](https://mlsanity.com/paper/2402.03994/) (2024)
- [Faithful and Efficient Explanations for Neural Networks via Neural Tangent Kernel Surrogate Models](https://mlsanity.com/paper/2305.14585/) (2023)
- [Influence Dynamics and Stagewise Data Attribution](https://mlsanity.com/paper/2510.12071/) (2025)
- [Final-Model-Only Data Attribution with a Unifying View of Gradient-Based Methods](https://mlsanity.com/paper/2412.03906/) (2024)
- [Learning to Weight Parameters for Training Data Attribution](https://mlsanity.com/paper/2506.05647/) (2025)

## Built on by (most cited first, 1 of 1)

- [More Data, Worse Decisions? Preference Reversals in Neural Networks under Gram Incompatibility](https://mlsanity.com/paper/2607.27255/) (2026)

## Abstract (arXiv)

Neural networks increasingly inform consequential decisions, making their reliability increasingly important. Yet their internal mechanisms provide little evidence of whether decisions remain grounded in the training cases and which cases ultimately support or oppose their outcomes. Without this connection between decisions and training cases, users cannot determine whether a model has learned reliable decision patterns from data. This motivates a fundamental question: do neural networks preserve case structure? We establish a connection between neural networks and Case-Based Decision Theory (CBDT), showing that trained neural networks can preserve a recoverable case structure through their learned representations. Such a structure allows fitted decision margins to be decomposed into individual case contributions. We identify the conditions under which this recovered case structure admits a CBDT interpretation. We further establish decision consistency between this interpretation and the decision selected by the original neural network. Experiments on a controlled CBDT setting and three decision tasks based on real-world data validate our approach. These results connect neural network decisions with the cases that shape them. This connection allows model choices to be traced back to supporting and opposing cases, providing a basis for assessing the reliability of neural network decisions.