# Decoupled Rationalization with Asymmetric Learning Rates: A Flexible Lipschitz Restraint

arXiv: https://arxiv.org/abs/2305.13599 · HTML: https://mlsanity.com/paper/2305.13599/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 45 references that are in the map.
- First posted on arXiv 23 May 2023.

## Builds on (most cited first, 30 of 32)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Lipschitz regularity of deep neural networks: analysis and efficient estimation](https://mlsanity.com/paper/1805.10965/) (2018)
- [Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks](https://mlsanity.com/paper/1906.04893/) (2019)
- [Rationalizing Neural Predictions](https://mlsanity.com/paper/1606.04155/) (2016)
- [Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach](https://mlsanity.com/paper/1801.10578/) (2018)
- [Invariant Rationalization](https://mlsanity.com/paper/2003.09772/) (2020)
- [Interpretable Neural Predictions with Differentiable Binary Variables](https://mlsanity.com/paper/1905.08160/) (2019)
- [Learning to Faithfully Rationalize by Construction](https://mlsanity.com/paper/2005.00115/) (2020)
- [Rethinking Cooperative Rationalization: Introspective Extraction and Complement Control](https://mlsanity.com/paper/1910.13294/) (2019)
- [An Information Bottleneck Approach for Controlling Conciseness in Rationale Extraction](https://mlsanity.com/paper/2005.00652/) (2020)
- [Deriving Machine Attention from Human Rationales](https://mlsanity.com/paper/1808.09367/) (2018)
- [Rethinking Lipschitz Neural Networks and Certified Robustness: A Boolean Function Perspective](https://mlsanity.com/paper/2210.01787/) (2022)
- [Bi-level Actor-Critic for Multi-agent Coordination](https://mlsanity.com/paper/1909.03510/) (2019)
- [Understanding Interlocking Dynamics of Cooperative Rationalization](https://mlsanity.com/paper/2110.13880/) (2021)
- [Can Rationalization Improve Robustness?](https://mlsanity.com/paper/2204.11790/) (2022)
- [A Game Theoretic Approach to Class-wise Selective Rationalization](https://mlsanity.com/paper/1910.12853/) (2019)
- [UNIREX: A Unified Learning Framework for Language Model Rationale Extraction](https://mlsanity.com/paper/2112.08802/) (2021)
- [Cooperative Learning of Disjoint Syntax and Semantics](https://mlsanity.com/paper/1902.09393/) (2019)
- [Gradient Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/2109.02235/) (2021)
- [Lipschitz Generative Adversarial Nets](https://mlsanity.com/paper/1902.05687/) (2019)
- [Learning to Scaffold: Optimizing Model Explanations for Teaching](https://mlsanity.com/paper/2204.10810/) (2022)
- [FR: Folded Rationalization with a Unified Encoder](https://mlsanity.com/paper/2209.08285/) (2022)
- [The Irrationality of Neural Rationale Models](https://mlsanity.com/paper/2110.07550/) (2021)
- [Distribution Matching for Rationalization](https://mlsanity.com/paper/2106.00320/) (2021)

## Built on by (most cited first, 14 of 14)

- [D-Separation for Causal Self-Explanation](https://mlsanity.com/paper/2309.13391/) (2023)
- [Multi-Level Explanations for Generative Language Models](https://mlsanity.com/paper/2403.14459/) (2024)
- [Is the MMI Criterion Necessary for Interpretability? Degenerating Non-causal Features to Plain Noise for Self-Rationalization](https://mlsanity.com/paper/2410.06003/) (2024)
- [Breaking Free from MMI: A New Frontier in Rationalization by Probing Input Utilization](https://mlsanity.com/paper/2503.06202/) (2025)
- [Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets](https://mlsanity.com/paper/2505.02118/) (2025)
- [MARE: Multi-Aspect Rationale Extractor on Unsupervised Rationale Extraction](https://mlsanity.com/paper/2410.03531/) (2024)
- [PDE+: Enhancing Generalization via PDE with Adaptive Distributional Diffusion](https://mlsanity.com/paper/2305.15835/) (2023)
- [CAVE: Controllable Authorship Verification Explanations](https://mlsanity.com/paper/2406.16672/) (2024)
- [Graph-Guided Textual Explanation Generation Framework](https://mlsanity.com/paper/2412.12318/) (2024)
- [Boosting Explainability through Selective Rationalization in Pre-trained Language Models](https://mlsanity.com/paper/2501.03182/) (2025)
- [Adversarial Attack for Explanation Robustness of Rationalization Models](https://mlsanity.com/paper/2408.10795/) (2024)
- [Learnable Game-theoretic Policy Optimization for Data-centric Self-explanation Rationalization](https://mlsanity.com/paper/2510.13393/) (2025)
- [Interlocking-free Selective Rationalization Through Genetic-based Learning](https://mlsanity.com/paper/2412.10312/) (2024)
- [Enhancing Graph Neural Networks: A Mutual Learning Approach](https://mlsanity.com/paper/2510.19223/) (2025)

## Abstract (arXiv)

A self-explaining rationalization model is generally constructed by a cooperative game where a generator selects the most human-intelligible pieces from the input text as rationales, followed by a predictor that makes predictions based on the selected rationales. However, such a cooperative game may incur the degeneration problem where the predictor overfits to the uninformative pieces generated by a not yet well-trained generator and in turn, leads the generator to converge to a sub-optimal model that tends to select senseless pieces. In this paper, we theoretically bridge degeneration with the predictor's Lipschitz continuity. Then, we empirically propose a simple but effective method named DR, which can naturally and flexibly restrain the Lipschitz constant of the predictor, to address the problem of degeneration. The main idea of DR is to decouple the generator and predictor to allocate them with asymmetric learning rates. A series of experiments conducted on two widely used benchmarks have verified the effectiveness of the proposed method. Codes: hrefhttps://github.com/jugechengzi/Rationalization-DRhttps://github.com/jugechengzi/Rationalization-DR.