# Think Through a Bottleneck: Hourglass Reasoning for Rigorous Induction

arXiv: https://arxiv.org/abs/2607.11696 · HTML: https://mlsanity.com/paper/2607.11696/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 33 of its 56 references that are in the map.
- First posted on arXiv 13 Jul 2026.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 30 of 33)

- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [Large Language Models Can Be Easily Distracted by Irrelevant Context](https://mlsanity.com/paper/2302.00093/) (2023)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs](https://mlsanity.com/paper/2506.14245/) (2025)
- [Hyperspherical Variational Auto-Encoders](https://mlsanity.com/paper/1804.00891/) (2018)
- [Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory](https://mlsanity.com/paper/2504.07952/) (2025)
- [Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models](https://mlsanity.com/paper/2401.01301/) (2024)
- [BIG-Bench Extra Hard](https://mlsanity.com/paper/2502.19187/) (2025)
- [SimpleToM: Exposing the Gap between Explicit ToM Inference and Implicit ToM Application in LLMs](https://mlsanity.com/paper/2410.13648/) (2024)
- [Comparing Humans, GPT-4, and GPT-4V On Abstraction and Reasoning Tasks](https://mlsanity.com/paper/2311.09247/) (2023)
- [Leveraging Language to Learn Program Abstractions and Search Heuristics](https://mlsanity.com/paper/2106.11053/) (2021)
- [Understanding the Dark Side of LLMs' Intrinsic Self-Correction](https://mlsanity.com/paper/2412.14959/) (2024)
- [BottleSum: Unsupervised and Self-supervised Sentence Summarization using the Information Bottleneck Principle](https://mlsanity.com/paper/1909.07405/) (2019)
- [Specializing Word Embeddings (for Parsing) by Information Bottleneck](https://mlsanity.com/paper/1910.00163/) (2019)
- [DeepRTL: Bridging Verilog Understanding and Generation with a Unified Representation Model](https://mlsanity.com/paper/2502.15832/) (2025)
- [Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Models](https://mlsanity.com/paper/2507.02778/) (2025)
- [Revisiting LLM Reasoning via Information Bottleneck](https://mlsanity.com/paper/2507.18391/) (2025)
- [Product of Experts with LLMs: Boosting Performance on ARC Is a Matter of Perspective](https://mlsanity.com/paper/2505.07859/) (2025)
- [IOLBENCH: Benchmarking LLMs on Linguistic Reasoning](https://mlsanity.com/paper/2501.04249/) (2025)
- [ChipBench: A Next-Step Benchmark for Evaluating LLM Performance in AI-Aided Chip Design](https://mlsanity.com/paper/2601.21448/) (2026)
- [SkillMaster: Toward Autonomous Skill Mastery in LLM Agents](https://mlsanity.com/paper/2605.08693/) (2026)
- [LingBench++: A Linguistically-Informed Benchmark and Reasoning Framework for Multi-Step and Cross-Cultural Inference with LLMs](https://mlsanity.com/paper/2507.16809/) (2025)

## Abstract (arXiv)

Self-refinement often fails to strengthen few-shot inductive reasoning in large language models. Prompting a model to explicitly state its inferred rule does little on its own. What actually matters is a structurally enforced isolation between reasoning stages, so that information can only pass between them as a compressed symbolic state. We introduce Hourglass reasoning, which enforces strict context isolation between reasoning stages. The frozen LLM acts as a meta-constructor, building for each task a symbolic encoder--decoder: an Induction module compresses the support examples into a schema φ (encoder) and a transient scaffold z; a Deduction module derives rule T (decoder) from these and discards z; an Implementer compiles (φ, T) into artifacts; an error-driven Refiner revises (φ, T) and regenerates artifacts from scratch. Only (φ, T) crosses stage boundaries, so all refinement stays anchored to the rule. We evaluate Hourglass across three benchmarks spanning visual abstraction, hardware synthesis, and textual rule induction, using GPT-5.5 and Gemini 3.1 Pro. On ARC-AGI-2, it raises best-of-5 accuracy by up to 14 points over an iterative-refinement baseline. On ChipBench, it nearly doubles Verilog synthesis accuracy with GPT-5.5, from 31% to 58%. BBEH-Linguini draws on puzzles from the International Linguistics Olympiad, a setting where prior work has shown that explicit verbalization can hurt performance. Hourglass mitigates this tendency, and on Gemini 3.1 Pro, it reverses the effect entirely. Ablations confirm that these gains come from the isolation between stages and the quality of the initial induction, not from prompt wording or the particular symbolic form used. It is how information flows through the reasoning process, rather than the language used to express it, that drives inductive reasoning in frozen LLMs.