mlsanityData to 7 Oct 2026

TEFM: Token-Efficient Faithful Modeling for Structured Data

arXiv 2609.09552 ↗First posted 9 Sep 2026cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper, we solve two fundamental obstacles in applying LLMs to critical domains: token efficiency and faithfulness. To address both constraints jointly, we present TEFM (Token-Efficient Faithful Modeling), a framework designed for structured data analysis in critical domains. TEFM achieves token efficiency by compressing lengthy structured observations into compact Behavioral Code tokens, dramatically reducing token consumption with minimal information loss. Moreover, TEFM enables faithful rationalization through a dual-fidelity objective that jointly optimizes code-level reconstruction and prediction-level fidelity, identifying minimal sufficient feature subsets grounded in input data. Comprehensive experiments across various domain datasets and model backbones (Qwen3, Gemma-2, Phi-4) show that TEFM achieves competitive classification accuracy with dramatic token reduction (approximately 1% token retention in clinical and 2% in security domains) while producing faithful rationales.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 24 references that are in the map. First posted on arXiv 9 Sep 2026.

Builds on

15 of 24 references.

References · 15

  1. 2017Neural Discrete Representation Learningcited by 4,650
  2. 2020Longformer: The Long-Document Transformercited by 3,362
  3. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  4. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  5. 2020Reformer: The Efficient Transformercited by 1,658
  6. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  7. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 699
  8. 2020Efficient Content-Based Sparse Attention with Routing Transformerscited by 469
  9. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  10. 2016Rationalizing Neural Predictionscited by 281
  11. 2018Scalable and accurate deep learning for electronic health recordscited by 275
  12. 2023Recommender Systems with Generative Retrievalcited by 191
3 more
  1. 2020WT5?! Training Text-to-Text Models to Explain their Predictionscited by 148
  2. 2019Interpretable Neural Predictions with Differentiable Binary Variablescited by 126
  3. 2022Learning Vector-Quantized Item Representation for Transferable Sequential Recommenderscited by 112

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026TrustDABench: Benchmarking Reliability and Robustness of LLMs for Structured Data Analysis
  2. 2026Domain-Specific Data Synthesis for LLMs via Minimal Sufficient Representation Learning
  3. 2025Predictive Auditing of Hidden Tokens in LLM APIs via Reasoning Length Estimation
  4. 2025On multi-token prediction for efficient LLM inference
  5. 2026TAME: Token Attribution and Masking for Emergent misalignment
  6. 2025TECP: Token-Entropy Conformal Prediction for LLMs
  7. 2026Few Tokens, Big Leverage: Preserving Safety Alignment by Constraining Safety Tokens during Fine-tuning
  8. 2026Rendering Data Unlearnable by Exploiting LLM Alignment Mechanisms