# On the Monitorability of Natural Language Autoencoders: Identification vs Discrimination

arXiv: https://arxiv.org/abs/2610.18727 · HTML: https://mlsanity.com/paper/2610.18727/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 33 of its 43 references that are in the map.
- First posted on arXiv 13 Oct 2026.

## Builds on (most cited first, 30 of 33)

- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [Multi-Agent Cooperation and the Emergence of (Natural) Language](https://mlsanity.com/paper/1612.07182/) (2016)
- [Emergent Multi-Agent Communication in the Deep Learning Era](https://mlsanity.com/paper/2006.02419/) (2020)
- [A Theory of Usable Information Under Computational Constraints](https://mlsanity.com/paper/2002.10689/) (2020)
- [Auditing language models for hidden objectives](https://mlsanity.com/paper/2503.10965/) (2025)
- [Off-Belief Learning](https://mlsanity.com/paper/2103.04000/) (2021)
- [On the Pitfalls of Measuring Emergent Communication](https://mlsanity.com/paper/1903.05168/) (2019)
- [Multi-agent Communication meets Natural Language: Synergies between Functional and Structural Language Learning](https://mlsanity.com/paper/2005.07064/) (2020)
- [Eliciting Latent Knowledge from Quirky Language Models](https://mlsanity.com/paper/2312.01037/) (2023)
- [Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers](https://mlsanity.com/paper/2512.15674/) (2025)
- [Emergent Communication of Generalizations](https://mlsanity.com/paper/2106.02668/) (2021)
- [AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors](https://mlsanity.com/paper/2602.22755/) (2026)
- [Eliciting Secret Knowledge from Language Models](https://mlsanity.com/paper/2510.01070/) (2025)
- [Liars' Bench: Evaluating Lie Detectors for Language Models](https://mlsanity.com/paper/2511.16035/) (2025)
- [Benchmarking Deception Probes via Black-to-White Performance Boosts](https://mlsanity.com/paper/2507.12691/) (2025)
- [LLMs Can Covertly Sandbag on Capability Evaluations Against Chain-of-Thought Monitoring](https://mlsanity.com/paper/2508.00943/) (2025)
- [CTRL-ALT-DECEIT: Sabotage Evaluations for Automated AI R&D](https://mlsanity.com/paper/2511.09904/) (2025)
- [Towards More Human-like AI Communication: A Review of Emergent Communication Research](https://mlsanity.com/paper/2308.02541/) (2023)
- [Emergent Language: A Survey and Taxonomy](https://mlsanity.com/paper/2409.02645/) (2024)
- [ReferentialGym: A Nomenclature and Framework for Language Emergence & Grounding in (Visual) Referential Games](https://mlsanity.com/paper/2012.09486/) (2020)
- [Do Activation Verbalization Methods Convey Privileged Information?](https://mlsanity.com/paper/2509.13316/) (2025)
- [Analyzing and Improving Chain-of-Thought Monitorability Through Information Theory](https://mlsanity.com/paper/2602.18297/) (2026)
- [Interpretability Guarantees with Merlin-Arthur Classifiers](https://mlsanity.com/paper/2206.00759/) (2022)
- [One Probe Won't Catch Them All: Towards Targeted Deception Detection](https://mlsanity.com/paper/2602.01425/) (2026)
- [A Decision-Theoretic Formalisation of Steganography With Applications to LLM Monitoring](https://mlsanity.com/paper/2602.23163/) (2026)
- [MonitorBench: A Comprehensive Benchmark for Chain-of-Thought Monitorability in Large Language Models](https://mlsanity.com/paper/2603.28590/) (2026)
- [The Model Organism Lottery: Model Organism Interpretability Strongly Depends on Training Methodology](https://mlsanity.com/paper/2607.01033/) (2026)
- [Pando: Do Interpretability Methods Work When Models Won't Explain Themselves?](https://mlsanity.com/paper/2604.11061/) (2026)
- [Training Deliberative Monitors for Black-Box Scheming Detection](https://mlsanity.com/paper/2605.29601/) (2026)
- ["Did you lie?" Evaluating Lie Detectors across Model Scale and Belief-Verified Model Organisms](https://mlsanity.com/paper/2606.12618/) (2026)

## Abstract (arXiv)

Natural Language Autoencoders (NLAs) are proposed as oversight tools for language models: an Activation Verbaliser (AV) writes an English explanation of an activation vector, and an Activation Reconstructor (AR) maps the explanation back to a vector. Their value for oversight depends on a human monitor being able to tell a true account of an activation from a false one, given the Av's explanation. We formalise NLA training and NLA oversight deployment as two different referential games from Emergent Communication: training is a generative referential game, while oversight is a discriminative, multi-modal, partially-observable referential game played over a many-to-many relation between the monitored language model's activations and the accounts that are true of them. The gap between the two games predicts that AV's explanations will identify which activation they describe without discriminating its true account from closely matched false ones, since training never asks explanations to rule anything out. Using the NLA's own AR as a maximally-sympathetic surrogate monitor model, we experimentally confirm this theoretical insight on one released checkpoint: true accounts beat accounts of other activations in all 320 comparisons, but beat fluent rivals differing in a single claim only 60.2% of the time (std.err. 6.7%, chance at 50%, n = 35). A meaning-preserving rewrite by an uninvolved model costs only 2.1% of the distance to another activation's account, so the failure is not due to the NLA pair having developed private conventions in the AV's explanations. Our rival accounts test only transcript-level truths, which the monitored language model's conversation settles, so these results bound the easiest case and leave claims about the monitored model's own processing untested. We state conjectures that would make both results consequences of the framework.