The Bicameral Model: Bidirectional Hidden-State Coupling Between Parallel Language Models
Not on the map. Why?
Abstract (arXiv)
Existing multi-model and tool-augmented systems communicate by generating text, serializing every exchange through the output vocabulary. Can two pretrained language models instead coordinate through a continuous, concurrent channel? The Bicameral Model couples two frozen language models through a trainable neural interface on their intermediate hidden states. At every generation step, both models run in lockstep: a primary model drives the task while an auxiliary model operates tools, solves constraints, or executes code, with both conditioning on each other's activations through a translation network and a learned suppression gate (~1% of combined parameters). The gate learns a selective communication protocol from task loss alone, without a prescribed format. We demonstrate the mechanism across three tool backends. On arithmetic, coupling two 0.5B models with a calculator raises accuracy from 36% to 96%. On logic grid puzzles, coupling two 0.6B models with a Z3 solver achieves 1.7× the unaugmented baseline on ZebraLogic. On mathematical reasoning, coupling with a Python sandbox enables the auxiliary to generate problem-specific code from hidden-state signals alone, without ever seeing the problem text.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 42 of its 49 references that are in the map. First posted on arXiv 11 May 2026.
Builds on
42 of 49 references.
References · 42
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2025Qwen3 Technical Reportcited by 8,195
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
- 2024Mixtral of Expertscited by 1,785
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
30 more
- 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
- 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
- 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
- 2022Editing Models with Task Arithmeticcited by 1,076
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
- 2023Steering Language Models With Activation Engineeringcited by 824
- 2023TIES-Merging: Resolving Interference When Merging Modelscited by 795
- 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
- 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 679
- 2024Scaling and evaluating sparse autoencoderscited by 530
- 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
- 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 379
- 2024The Platonic Representation Hypothesiscited by 362
- 2023Function Vectors in Large Language Modelscited by 292
- 2023ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solvingcited by 276
- 2023ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddingscited by 238
- 2025ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoningcited by 186
- 2023LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Proverscited by 161
- 2023SatLM: Satisfiability-Aided Language Models Using Declarative Promptingcited by 100
- 2024Learning to Decode Collaboratively with Multiple Language Modelscited by 61
- 2025Cache-to-Cache: Direct Semantic Communication Between Large Language Modelscited by 54
- 2024FuseChat: Knowledge Fusion of Chat Modelscited by 53
- 2024LLM Augmented LLMs: Expanding Capabilities through Compositioncited by 35
- 2025Enabling Agents to Communicate Entirely in Latent Spacecited by 31
- 2025Thought Communication in Multiagent Collaborationcited by 28
- 2025Communicating Activations Between Language Model Agentscited by 15
- 2026Recursive Multi-Agent Systemscited by 10
- 2025Generalized Parallel Scaling with Interdependent Generationscited by 6
- 2025Improving Rule-based Reasoning in LLMs using Neurosymbolic Representationscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Can Language Models Follow Multiple Turns of Entangled Instructions?
- 2026A Negative Result on Cross-Model Activation Transfer in a Pythia Multi-Hop Setting
- 2025PromptBridge: Cross-Model Prompt Transfer for Large Language Models
- 2025Understanding Tool-Integrated Reasoning
- 2026Opening the Black Box: A Survey on the Mechanisms of Multi-Step Reasoning in Large Language Models
- 2026Harnessing LLMs as Agents: What Does It Cost?
- 2026VeriInteresting: An Empirical Study of Model Prompt Interactions in Verilog Code Generation
- 2025On the Limits of Innate Planning in Large Language Models