mlsanityData to 14 Oct 2026

Machine-Interpretable Information: Compiling Documents into Searchable and Readable Protocol States

arXiv 2609.23371 ↗First posted 20 Sep 2026cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Long-context language models interface with external knowledge through raw natural language. In retrieval-augmented systems, this creates a persistent index-payload schism: dense vectors enable searchable routing, but models must re-ingest lengthy text payloads for reasoning at O(N^2) attention cost. Existing compression methods further produce private states tied to specific architectures. We introduce Machine-Interpretable Information (MII), the first agent-to-agent (A2A) document-to-state protocol. A dual-timescale state-space Writer compiles documents into a canonical, fixed-bandwidth state (56 tokens), and a lightweight Translator maps it into any frozen Reader's embedding space, reducing query-time cost to O(K). The resulting .mii artifact unifies Retrieval (searchable geometry), Reasoning (global memory), and Reconstruction (grounded details) in a single transferable medium. We demonstrate strong cross-model interoperability across heterogeneous LLMs (e.g., Llama, Qwen, Mistral) -- despite the Writer using a legacy GPT-2 vocabulary, forcing genuine semantic translation rather than token-level memorization. Mechanistic probes reveal modular latent structure: entity representations can be causally traced and zero-shot transplanted between unrelated document states while remaining decodable. To address lexical reconstruction under fixed bandwidth, we propose Residual-MII, a cache hierarchy combining compiled global memory with sparse local evidence. On HotpotQA (7,405 queries), Residual-MII exceeds full-context Exact Match at approximately 7% of the attention FLOPs, suggesting a paradigm shift toward compiled, transferable neural document formats.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 61 of its 67 references that are in the map. First posted on arXiv 20 Sep 2026.

Builds on

61 of 67 references.

References · 61

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  6. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  7. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  8. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  9. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  10. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  11. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  12. 2020Longformer: The Long-Document Transformercited by 3,413
38 more
  1. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  2. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  3. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  4. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  5. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  6. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  7. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  8. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  9. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  10. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  11. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  12. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  13. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  14. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  15. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  16. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  17. 2022Mass-Editing Memory in a Transformercited by 1,001
  18. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
  19. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  20. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  21. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  22. 2020Efficient Transformers: A Surveycited by 790
  23. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 740
  24. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 726
  25. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  26. 2023In-Context Retrieval-Augmented Language Modelscited by 702
  27. 2014Distributed Representations of Sentences and Documentscited by 678
  28. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  29. 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 629
  30. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  31. 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 532
  32. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 517
  33. 2023Learning to Compress Prompts with Gist Tokenscited by 390
  34. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 356
  35. 2023Adapting Language Models to Compress Contextscited by 356
  36. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 327
  37. 2023L-Eval: Instituting Standardized Evaluation for Long Context Language Modelscited by 233
  38. 2021Revisiting Model Stitching to Compare Neural Representationscited by 209

50 most cited of 61 All 61 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Memory-Orchestrated Semantic System (MOSS): An Auditable Agentic Memory Architecture
  2. 2026Retrieval as Reasoning: Self-Evolving Agent-Native Retrieval via LLM-Wiki
  3. 2026ByteRover: Agent-Native Memory Through LLM-Curated Hierarchical Context
  4. 2026ObjectGraph: From Document Injection to Knowledge Traversal -- A Native File Format for the Agentic Era
  5. 2026Toward a Theory of Hierarchical Memory for Language Agents
  6. 2026Semantics Delivery Network: Rethinking Web Retrieval Infrastructure for LLM Agents
  7. 2026Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations
  8. 2026MemCode: Discrete Semantic Representations for Long-Term Agent Memory