# FR: Folded Rationalization with a Unified Encoder

arXiv: https://arxiv.org/abs/2209.08285 · HTML: https://mlsanity.com/paper/2209.08285/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 27 references that are in the map.
- First posted on arXiv 17 Sep 2022.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Rationalizing Neural Predictions](https://mlsanity.com/paper/1606.04155/) (2016)
- [Techniques for Interpretable Machine Learning](https://mlsanity.com/paper/1808.00033/) (2018)
- [Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning](https://mlsanity.com/paper/1702.08811/) (2017)
- [Invariant Rationalization](https://mlsanity.com/paper/2003.09772/) (2020)
- [Interpretable Neural Predictions with Differentiable Binary Variables](https://mlsanity.com/paper/1905.08160/) (2019)
- [Learning to Faithfully Rationalize by Construction](https://mlsanity.com/paper/2005.00115/) (2020)
- [Rethinking Cooperative Rationalization: Introspective Extraction and Complement Control](https://mlsanity.com/paper/1910.13294/) (2019)
- [Deriving Machine Attention from Human Rationales](https://mlsanity.com/paper/1808.09367/) (2018)
- [Understanding Interlocking Dynamics of Cooperative Rationalization](https://mlsanity.com/paper/2110.13880/) (2021)
- [A Game Theoretic Approach to Class-wise Selective Rationalization](https://mlsanity.com/paper/1910.12853/) (2019)
- [Learning from the Best: Rationalizing Prediction by Adversarial Information Calibration](https://mlsanity.com/paper/2012.08884/) (2020)
- [Interpreting Deep Learning Models in Natural Language Processing: A Review](https://mlsanity.com/paper/2110.10470/) (2021)
- [Distribution Matching for Rationalization](https://mlsanity.com/paper/2106.00320/) (2021)
- [Interpretable Complex-Valued Neural Networks for Privacy Protection](https://mlsanity.com/paper/1901.09546/) (2019)

## Built on by (most cited first, 18 of 18)

- [Self-Evolving Multi-Agent Systems via Textual Backpropagation](https://mlsanity.com/paper/2506.09046/) (2025)
- [D-Separation for Causal Self-Explanation](https://mlsanity.com/paper/2309.13391/) (2023)
- [Towards Trustworthy Explanation: On Causal Rationalization](https://mlsanity.com/paper/2306.14115/) (2023)
- [Enhancing the Rationale-Input Alignment for Self-explaining Rationalization](https://mlsanity.com/paper/2312.04103/) (2023)
- [Decoupled Rationalization with Asymmetric Learning Rates: A Flexible Lipschitz Restraint](https://mlsanity.com/paper/2305.13599/) (2023)
- [MGR: Multi-generator Based Rationalization](https://mlsanity.com/paper/2305.04492/) (2023)
- [Multi-Level Explanations for Generative Language Models](https://mlsanity.com/paper/2403.14459/) (2024)
- [Is the MMI Criterion Necessary for Interpretability? Degenerating Non-causal Features to Plain Noise for Self-Rationalization](https://mlsanity.com/paper/2410.06003/) (2024)
- [Breaking Free from MMI: A New Frontier in Rationalization by Probing Input Utilization](https://mlsanity.com/paper/2503.06202/) (2025)
- [Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets](https://mlsanity.com/paper/2505.02118/) (2025)
- [GNN Explanations that do not Explain and How to find Them](https://mlsanity.com/paper/2601.20815/) (2026)
- [MARE: Multi-Aspect Rationale Extractor on Unsupervised Rationale Extraction](https://mlsanity.com/paper/2410.03531/) (2024)
- [You Only Forward Once: Prediction and Rationalization in A Single Forward Pass](https://mlsanity.com/paper/2311.02344/) (2023)
- [Boosting Explainability through Selective Rationalization in Pre-trained Language Models](https://mlsanity.com/paper/2501.03182/) (2025)
- [X-Troll: eXplainable Detection of State-Sponsored Information Operations Agents](https://mlsanity.com/paper/2508.16021/) (2025)
- [Text Rationalization for Robust Causal Effect Estimation](https://mlsanity.com/paper/2512.05373/) (2025)
- [CMA-R:Causal Mediation Analysis for Explaining Rumour Detection](https://mlsanity.com/paper/2402.08155/) (2024)
- [Learnable Game-theoretic Policy Optimization for Data-centric Self-explanation Rationalization](https://mlsanity.com/paper/2510.13393/) (2025)

## Abstract (arXiv)

Conventional works generally employ a two-phase model in which a generator selects the most important pieces, followed by a predictor that makes predictions based on the selected pieces. However, such a two-phase model may incur the degeneration problem where the predictor overfits to the noise generated by a not yet well-trained generator and in turn, leads the generator to converge to a sub-optimal model that tends to select senseless pieces. To tackle this challenge, we propose Folded Rationalization (FR) that folds the two phases of the rationale model into one from the perspective of text semantic extraction. The key idea of FR is to employ a unified encoder between the generator and predictor, based on which FR can facilitate a better predictor by access to valuable information blocked by the generator in the traditional two-phase model and thus bring a better generator. Empirically, we show that FR improves the F1 score by up to 10.3% as compared to state-of-the-art methods.