# SparCAssist: A Model Risk Assessment Assistant Based on Sparse Generated Counterfactuals

arXiv: https://arxiv.org/abs/2205.01588 · HTML: https://mlsanity.com/paper/2205.01588/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 39 references that are in the map.
- First posted on arXiv 3 May 2022.

## Builds on (most cited first, 24 of 24)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [GPT Understands, Too](https://mlsanity.com/paper/2103.10385/) (2021)
- [ERASER: A Benchmark to Evaluate Rationalized NLP Models](https://mlsanity.com/paper/1911.03429/) (2019)
- [Learning the Difference that Makes a Difference with Counterfactually-Augmented Data](https://mlsanity.com/paper/1909.12434/) (2019)
- [AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts](https://mlsanity.com/paper/2010.15980/) (2020)
- [Explainable Machine Learning in Deployment](https://mlsanity.com/paper/1909.06342/) (2019)
- [Explaining NLP Models via Minimal Contrastive Editing (MiCE)](https://mlsanity.com/paper/2012.13985/) (2020)
- [Imitation Attacks and Defenses for Black-box Machine Translation Systems](https://mlsanity.com/paper/2004.15015/) (2020)
- [Explanation-Based Human Debugging of NLP Models: A Survey](https://mlsanity.com/paper/2104.15135/) (2021)
- [When is invariance useful in an Out-of-Distribution Generalization problem ?](https://mlsanity.com/paper/2008.01883/) (2020)
- [Zorro: Valid, Sparse, and Stable Explanations in Graph Neural Networks](https://mlsanity.com/paper/2105.08621/) (2021)
- [Explain and Predict, and then Predict Again](https://mlsanity.com/paper/2101.04109/) (2021)
- [EXS: Explainable Search Using Local Model Agnostic Interpretability](https://mlsanity.com/paper/1809.03857/) (2018)
- [The Irrationality of Neural Rationale Models](https://mlsanity.com/paper/2110.07550/) (2021)
- [Posthoc Interpretability of Learning to Rank Models using Secondary Training Data](https://mlsanity.com/paper/1806.11330/) (2018)
- [Towards Benchmarking the Utility of Explanations for Model Debugging](https://mlsanity.com/paper/2105.04505/) (2021)
- [Tempas: Temporal Archive Search Based on Tags](https://mlsanity.com/paper/1702.01076/) (2017)
- [Discovering Entities with Just a Little Help from You](https://mlsanity.com/paper/1810.10252/) (2018)
- [FaxPlainAC: A Fact-Checking Tool Based on EXPLAINable Models with HumAn Correction in the Loop](https://mlsanity.com/paper/2110.10144/) (2021)

## Built on by (most cited first, 1 of 1)

- [DISCO: DISCovering Overfittings as Causal Rules for Text Classification Models](https://mlsanity.com/paper/2411.04649/) (2024)

## Abstract (arXiv)

We introduce SparcAssist, a general-purpose risk assessment tool for the machine learning models trained for language tasks. It evaluates models' risk by inspecting their behavior on counterfactuals, namely out-of-distribution instances generated based on the given data instance. The counterfactuals are generated by replacing tokens in rational subsequences identified by ExPred, while the replacements are retrieved using HotFlip or Masked-Language-Model-based algorithms. The main purpose of our system is to help the human annotators to assess the model's risk on deployment. The counterfactual instances generated during the assessment are the by-product and can be used to train more robust NLP models in the future.