Bongard: Training Machine Intuition
Not on the map. Why?
Abstract (arXiv)
Human intelligence relies heavily on learned intuition: recognising patterns and judging situations without explicitly unfolding every intermediate step. We introduce Bongard, an open-weight System One model that treats machine intuition as an independent capability to design and train. A T5Gemma 2 4B-4B encoder-decoder separates reading the evidence from making judgments. The encoder reads the state bidirectionally together with the question instructions, and separate decoder branches share this encoding, so many judgments about the same situation require only one reading of the state. A trained head returns probabilities over the supplied candidates without generating text. Training proceeds in three stages, from supervised judgments to semantic relationships to action outcomes, and each stage updates all 7.09 billion trainable parameters on one Blackwell GPU. Joint-embedding post-training raises accuracy on held-out rephrasings from 75.7% to 85.9%. A sandbox stage then learns outcome distributions from action rollouts and exact oracles, raising accuracy on a frozen sandbox panel from 50.6% to 64.8%. On DecisionBench, the final model reaches 78.05% accuracy over 23,900 decisions and ranks fourth of 61 systems in the public comparison. On one RTX PRO 6000, its median latency is 36 ms for short requests, and 32 questions about one state take 221 ms. Bongard demonstrates that machine intuition can be systematically trained via representation learning and outcome feedback, providing an open, efficient alternative for high-throughput decision workloads.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 61 of its 96 references that are in the map. First posted on arXiv 30 Sep 2026.
Builds on
61 of 96 references.
References · 61
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
- 2021Program Synthesis with Large Language Modelscited by 3,637
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
- 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
- 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
- 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
- 2025Gemma 3 Technical Reportcited by 1,824
38 more
- 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
- 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
- 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
- 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
- 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
- 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,209
- 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
- 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
- 2016Training Deep Nets with Sublinear Memory Costcited by 848
- 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
- 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 784
- 2017Crowdsourcing Multiple Choice Science Questionscited by 765
- 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 531
- 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 486
- 2020Efficient Intent Detection with Dual Sentence Encoderscited by 433
- 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 431
- 2021Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depthcited by 419
- 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 417
- 20218-bit Optimizers via Block-wise Quantizationcited by 403
- 2019PAWS: Paraphrase Adversaries from Word Scramblingcited by 402
- 2015Deep Learning with Limited Numerical Precisioncited by 376
- 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 376
- 2024JudgeBench: A Benchmark for Evaluating LLM-based Judgescited by 318
- 2022UL2: Unifying Language Learning Paradigmscited by 295
- 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 283
- 2024From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Stepcited by 235
- 2023ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversationcited by 231
- 2022FP8 Formats for Deep Learningcited by 196
- 2024Flex Attention: A Programming Model for Generating Optimized Attention Kernelscited by 185
- 2016ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learningcited by 179
- 2022MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languagescited by 146
- 2025RWKV-7 "Goose" with Expressive Dynamic State Evolutioncited by 141
- 2025Beyond Binary Rewards: Training LMs to Reason About Their Uncertaintycited by 117
- 2022RankT5: Fine-Tuning T5 for Text Ranking with Ranking Lossescited by 115
- 2021Calibrating Predictions to Decisions: A Novel Approach to Multi-Class Calibrationcited by 79
- 2025Pretraining Large Language Models with NVFP4cited by 76
- 2024Hydragen: High-Throughput LLM Inference with Shared Prefixescited by 73
- 2026FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scalingcited by 66
50 most cited of 61 All 61 (JSON)
Built on by
Not cited yet.