mlsanityData to 7 Oct 2026

For What Reason? Interpreting Models' Encoding of Causation and Antithesis

arXiv 2607.18570 ↗First posted 20 Jul 2026cs.CL cs.AIcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Discourse relations provide document structure, critical to language understanding and enabling language model performance and ethicality. In this work, we investigate how instruction-tuned Transformer models (LLaMA and Mistral) encode discourse relations in English, with a particular focus on the contrasting relations of causation and antithesis. Framing the task as a next-token prediction task and applying a suite of interpretability techniques to test model internals, our findings show that certain early layers make predictive decisions at mid-sequence tokens, while some mid-level layers finalize their decisions closer to the last token. Most of the remaining layers primarily propagate earlier decisions rather than actively influencing them. Additionally, we observe that some layers exhibit a preference for one answer over alternatives, suggesting asymmetric representation of discourse-based reasoning.footnoteOur code is available at https://github.com/abhidipbhattacharyya/causationvsantithesis

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 3 of its 40 references that are in the map. First posted on arXiv 20 Jul 2026.

Builds on

3 of 40 references.

References · 3

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2020Longformer: The Long-Document Transformercited by 3,362
  3. 2023Mistral 7Bcited by 3,065

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Base Models Know How to Reason, Thinking Models Learn When
  2. 2025WUGNECTIVES: Novel Entity Inferences of Language Models from Discourse Connectives
  3. 2026Knowing, and Saying It Only When Asked: LLM Endognostics and the Schizognosis of Minerva-7B
  4. 2026Causal Tongue-Tie: LLMs Can Encode Causal Direction, But Their Yes/No Outputs Fail to Express
  5. 2026Convergence Without Understanding: When Language Models Agree on Representations but Disagree on Reasoning
  6. 2025State over Tokens: Characterizing the Role of Reasoning Tokens
  7. 2026Tokengeist: Multi-Turn Attribution Tracing in Agentic Conversations
  8. 2024Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Models