# MGR: Multi-generator Based Rationalization

arXiv: https://arxiv.org/abs/2305.04492 · HTML: https://mlsanity.com/paper/2305.04492/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 38 references that are in the map.
- First posted on arXiv 8 May 2023.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Git Re-Basin: Merging Models modulo Permutation Symmetries](https://mlsanity.com/paper/2209.04836/) (2022)
- [Rationalizing Neural Predictions](https://mlsanity.com/paper/1606.04155/) (2016)
- [Model Fusion via Optimal Transport](https://mlsanity.com/paper/1910.05653/) (2019)
- [Invariant Rationalization](https://mlsanity.com/paper/2003.09772/) (2020)
- [Interpretable Neural Predictions with Differentiable Binary Variables](https://mlsanity.com/paper/1905.08160/) (2019)
- [Learning to Faithfully Rationalize by Construction](https://mlsanity.com/paper/2005.00115/) (2020)
- [Rethinking Cooperative Rationalization: Introspective Extraction and Complement Control](https://mlsanity.com/paper/1910.13294/) (2019)
- [Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language?](https://mlsanity.com/paper/2010.04119/) (2020)
- [An Information Bottleneck Approach for Controlling Conciseness in Rationale Extraction](https://mlsanity.com/paper/2005.00652/) (2020)
- [Deriving Machine Attention from Human Rationales](https://mlsanity.com/paper/1808.09367/) (2018)
- [SelfExplain: A Self-Explaining Architecture for Neural Text Classifiers](https://mlsanity.com/paper/2103.12279/) (2021)
- [Understanding Interlocking Dynamics of Cooperative Rationalization](https://mlsanity.com/paper/2110.13880/) (2021)
- [Can Rationalization Improve Robustness?](https://mlsanity.com/paper/2204.11790/) (2022)
- [A Game Theoretic Approach to Class-wise Selective Rationalization](https://mlsanity.com/paper/1910.12853/) (2019)
- [UNIREX: A Unified Learning Framework for Language Model Rationale Extraction](https://mlsanity.com/paper/2112.08802/) (2021)
- [SPECTRA: Sparse Structured Text Rationalization](https://mlsanity.com/paper/2109.04552/) (2021)
- [FR: Folded Rationalization with a Unified Encoder](https://mlsanity.com/paper/2209.08285/) (2022)
- [Distribution Matching for Rationalization](https://mlsanity.com/paper/2106.00320/) (2021)
- [Making a (Counterfactual) Difference One Rationale at a Time](https://mlsanity.com/paper/2201.05177/) (2022)

## Built on by (most cited first, 12 of 12)

- [D-Separation for Causal Self-Explanation](https://mlsanity.com/paper/2309.13391/) (2023)
- [Beyond Zero Initialization: Investigating the Impact of Non-Zero Initialization on LoRA Fine-Tuning Dynamics](https://mlsanity.com/paper/2505.23194/) (2025)
- [Decoupled Rationalization with Asymmetric Learning Rates: A Flexible Lipschitz Restraint](https://mlsanity.com/paper/2305.13599/) (2023)
- [Breaking Free from MMI: A New Frontier in Rationalization by Probing Input Utilization](https://mlsanity.com/paper/2503.06202/) (2025)
- [Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets](https://mlsanity.com/paper/2505.02118/) (2025)
- [The Panaceas for Improving Low-Rank Decomposition in Communication-Efficient Federated Learning](https://mlsanity.com/paper/2505.23176/) (2025)
- [Beyond External Monitors: Enhancing Transparency of Large Language Models for Easier Monitoring](https://mlsanity.com/paper/2502.05242/) (2025)
- [X-Troll: eXplainable Detection of State-Sponsored Information Operations Agents](https://mlsanity.com/paper/2508.16021/) (2025)
- [CMA-R:Causal Mediation Analysis for Explaining Rumour Detection](https://mlsanity.com/paper/2402.08155/) (2024)
- [Adversarial Attack for Explanation Robustness of Rationalization Models](https://mlsanity.com/paper/2408.10795/) (2024)
- [Learnable Game-theoretic Policy Optimization for Data-centric Self-explanation Rationalization](https://mlsanity.com/paper/2510.13393/) (2025)
- [Enhancing Graph Neural Networks: A Mutual Learning Approach](https://mlsanity.com/paper/2510.19223/) (2025)

## Abstract (arXiv)

Rationalization is to employ a generator and a predictor to construct a self-explaining NLP model in which the generator selects a subset of human-intelligible pieces of the input text to the following predictor. However, rationalization suffers from two key challenges, i.e., spurious correlation and degeneration, where the predictor overfits the spurious or meaningless pieces solely selected by the not-yet well-trained generator and in turn deteriorates the generator. Although many studies have been proposed to address the two challenges, they are usually designed separately and do not take both of them into account. In this paper, we propose a simple yet effective method named MGR to simultaneously solve the two problems. The key idea of MGR is to employ multiple generators such that the occurrence stability of real pieces is improved and more meaningful pieces are delivered to the predictor. Empirically, we show that MGR improves the F1 score by up to 20.9% as compared to state-of-the-art methods. Codes are available at https://github.com/jugechengzi/Rationalization-MGR .