# SOInter: A Novel Deep Energy Based Interpretation Method for Explaining Structured Output Models

arXiv: https://arxiv.org/abs/2202.09914 · HTML: https://mlsanity.com/paper/2202.09914/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 23 references that are in the map.
- First posted on arXiv 20 Feb 2022.

## Builds on (most cited first, 13 of 13)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Learning to Explain: An Information-Theoretic Perspective on Model Interpretation](https://mlsanity.com/paper/1802.07814/) (2018)
- [Visualizing Deep Neural Network Decisions: Prediction Difference Analysis](https://mlsanity.com/paper/1702.04595/) (2017)
- [Structured Prediction Energy Networks](https://mlsanity.com/paper/1511.06350/) (2015)
- [End-to-End Learning for Structured Prediction Energy Networks](https://mlsanity.com/paper/1703.05667/) (2017)
- [Fully Connected Deep Structured Networks](https://mlsanity.com/paper/1503.02351/) (2015)
- [Learning Deep Structured Models](https://mlsanity.com/paper/1407.2538/) (2014)
- [Deep Value Networks Learn to Evaluate and Iteratively Refine Structured Outputs](https://mlsanity.com/paper/1703.04363/) (2017)
- [Deep Structured Prediction with Nonlinear Output Transformations](https://mlsanity.com/paper/1811.00539/) (2018)

## Built on by (most cited first, 1 of 1)

- [Interpreting Black-Box Large Language Models with Sentence-Level Energy Landscapes](https://mlsanity.com/paper/2608.02879/) (2026)

## Abstract (arXiv)

We propose a novel interpretation technique to explain the behavior of structured output models, which learn mappings between an input vector to a set of output variables simultaneously. Because of the complex relationship between the computational path of output variables in structured models, a feature can affect the value of output through other ones. We focus on one of the outputs as the target and try to find the most important features utilized by the structured model to decide on the target in each locality of the input space. In this paper, we assume an arbitrary structured output model is available as a black box and argue how considering the correlations between output variables can improve the explanation performance. The goal is to train a function as an interpreter for the target output variable over the input space. We introduce an energy-based training process for the interpreter function, which effectively considers the structural information incorporated into the model to be explained. The effectiveness of the proposed method is confirmed using a variety of simulated and real data sets.