mlsanityData to 14 Oct 2026

StateSwap: Probing Support-Elimination Hidden States in Multiple-Choice Questions

arXiv 2609.01081 ↗First posted 1 Sep 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models often answer the same multiple-choice question inconsistently when it is posed under support-oriented and elimination-oriented framings. We investigate whether these discrepancies arise from different internal representations induced by the two framings. We introduce a dual-framing protocol with minimally varied prompts that use either support- or elimination-oriented framing while keeping the evaluation target fixed. To probe the internal computation, we append an untrained special token, [STATE], and treat its residual-stream activation as an intervention interface. Across both models, the two framings induce separable [STATE] activations concentrated in intermediate layers. Swapping these activations between paired prompts systematically changes predictions and improves cross-framing agreement, providing intervention-based evidence that the activations are behaviorally relevant. Beyond instance-level substitution, mean-difference steering directions derived from the dual-framing contrast exhibit more bounded layer-wise responses than matched contrastive activation addition directions under the evaluated protocol.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 31 references that are in the map. First posted on arXiv 1 Sep 2026.

Builds on

20 of 31 references.

References · 20

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  4. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  6. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  7. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  8. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  9. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  10. 2022In-context Learning and Induction Headscited by 964
  11. 2024How to use and interpret activation patchingcited by 214
  12. 2024Programming Refusal with Conditional Activation Steeringcited by 206
8 more
  1. 2025FairSteer: Inference Time Debiasing for LLMs with Dynamic Activation Steeringcited by 40
  2. 2025Latent Chain-of-Thought? Decoding the Depth-Recurrent Transformercited by 27
  3. 2025How does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencodingcited by 19
  4. 2023Just Ask One More Time! Self-Agreement Improves Reasoning of Language Models in (Almost) All Scenarioscited by 18
  5. 2025Finite State Automata Inside Transformers with Chain-of-Thought: A Mechanistic Study on State Trackingcited by 18
  6. 2023It's Not Easy Being Wrong: Large Language Models Struggle with Process of Elimination Reasoningcited by 12
  7. 2023POE: Process of Elimination for Multiple Choice Reasoningcited by 8
  8. 2025Option-ID Based Elimination For Multiple Choice Questionscited by 2

Built on by

Not cited yet.