# Cross-Loss Influence Functions to Explain Deep Network Representations

arXiv: https://arxiv.org/abs/2012.01685 · HTML: https://mlsanity.com/paper/2012.01685/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Dec 2020.

## Built on by (most cited first, 12 of 12)

- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [Evolving Interpretable Visual Classifiers with Large Language Models](https://mlsanity.com/paper/2404.09941/) (2024)
- [Sample based Explanations via Generalized Representers](https://mlsanity.com/paper/2310.18526/) (2023)
- [Towards Explainable Artificial Intelligence (XAI): A Data Mining Perspective](https://mlsanity.com/paper/2401.04374/) (2024)
- [Towards Balancing Preference and Performance through Adaptive Personalized Explainability](https://mlsanity.com/paper/2504.13856/) (2025)
- [Towards Reconciling Usability and Usefulness of Explainable AI Methodologies](https://mlsanity.com/paper/2301.05347/) (2023)
- [Detecting Instruction Fine-tuning Attacks using Influence Function](https://mlsanity.com/paper/2504.09026/) (2025)
- [Addressing Selection Bias in Computerized Adaptive Testing: A User-Wise Aggregate Influence Function Approach](https://mlsanity.com/paper/2308.11912/) (2023)
- [How Video Meetings Change Your Expression](https://mlsanity.com/paper/2406.00955/) (2024)
- [Targeted Fine-Tuning of DNN-Based Receivers via Influence Functions](https://mlsanity.com/paper/2509.15950/) (2025)
- [Price of Fairness in Short-Term and Long-Term Algorithmic Selections](https://mlsanity.com/paper/2605.06227/) (2026)
- [Understanding Curriculum Learning in Large Language Models via Cross-Difficulty Optimization Dynamics](https://mlsanity.com/paper/2608.17268/) (2026)

## Abstract (arXiv)

As machine learning is increasingly deployed in the real world, it is paramount that we develop the tools necessary to analyze the decision-making of the models we train and deploy to end-users. Recently, researchers have shown that influence functions, a statistical measure of sample impact, can approximate the effects of training samples on classification accuracy for deep neural networks. However, this prior work only applies to supervised learning, where training and testing share an objective function. No approaches currently exist for estimating the influence of unsupervised training examples for deep learning models. To bring explainability to unsupervised and semi-supervised training regimes, we derive the first theoretical and empirical demonstration that influence functions can be extended to handle mismatched training and testing (i.e., "cross-loss") settings. Our formulation enables us to compute the influence in an unsupervised learning setup, explain cluster memberships, and identify and augment biases in language models. Our experiments show that our cross-loss influence estimates even exceed matched-objective influence estimation relative to ground-truth sample impact.