mlsanityData to 14 Oct 2026

Advancing Sequential Numerical Prediction in Autoregressive Models

arXiv 2505.13077 ↗First posted 19 May 2025cs.CL cs.AI cs.LGcited by 26

Not on the map. Why?

Abstract (arXiv)

Autoregressive models have become the de facto choice for sequence generation tasks, but standard approaches treat digits as independent tokens and apply cross-entropy loss, overlooking the coherent structure of numerical sequences. This paper introduces Numerical Token Integrity Loss (NTIL) to address this gap. NTIL operates at two levels: (1) token-level, where it extends the Earth Mover's Distance (EMD) to preserve ordinal relationships between numerical values, and (2) sequence-level, where it penalizes the overall discrepancy between the predicted and actual sequences. This dual approach improves numerical prediction and integrates effectively with LLMs/MLLMs. Extensive experiments show significant performance improvements with NTIL.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Builds on 37 of its 54 references that are in the map. First posted on arXiv 19 May 2025.

Builds on

37 of 54 references.

References · 37

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  6. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  7. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  8. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
  9. 2024Qwen2 Technical Reportcited by 2,120
  10. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  11. 2016Modeling Context in Referring Expressionscited by 1,063
  12. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
25 more
  1. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 723
  2. 2024Yi: Open Foundation Models by 01.AIcited by 719
  3. 2024PaliGemma: A versatile 3B VLM for transfercited by 696
  4. 2023Baichuan 2: Open Large-scale Language Modelscited by 670
  5. 2017Wasserstein GANcited by 639
  6. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 526
  7. 2015Optimal Transport for Domain Adaptationcited by 463
  8. 2019Analysing Mathematical Reasoning Abilities of Neural Modelscited by 335
  9. 2017Total-Text: A Comprehensive Dataset for Scene Text Detection and Recognitioncited by 162
  10. 2020FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensionscited by 152
  11. 2023MAGE: Machine-generated Text Detection in the Wildcited by 132
  12. 2017Detecting Curve Text in the Wild: New Dataset and New Solutioncited by 83
  13. 2021A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learningcited by 82
  14. 2023Aligning Large Language Models with Human Preferences through Representation Engineeringcited by 71
  15. 2024A Bounding Box is Worth One Token: Interleaving Layout and Text in a Large Language Model for Document Understandingcited by 57
  16. 2016Squared Earth Mover's Distance-based Loss for Training Deep Neural Networkscited by 45
  17. 2023DocMath-Eval: Evaluating Math Reasoning Capabilities of LLMs in Understanding Long and Specialized Documentscited by 45
  18. 2025SCORE: Story Coherence and Retrieval Enhancement for AI Narrativescited by 42
  19. 2024KIEval: A Knowledge-grounded Interactive Evaluation Framework for Large Language Modelscited by 36
  20. 2024Multi-Level Optimal Transport for Universal Cross-Tokenizer Knowledge Distillation on Language Modelscited by 36
  21. 2022What Makes Pre-trained Language Models Better Zero-shot Learners?cited by 23
  22. 2023CopyNE: Better Contextual ASR by Copying Named Entitiescited by 14
  23. 2022PUnifiedNER: A Prompting-based Unified NER System for Diverse Datasetscited by 10
  24. 2024Uncovering the Full Potential of Visual Grounding Methods in VQAcited by 9
  25. 2024Muffin or Chihuahua? Challenging Multimodal Large Language Models with Multipanel VQAcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 23.

26 papers.

Sort:

By area:

  1. 2025Prolonged Reasoning Is Not All You Need: Certainty-Based Adaptive Routing for Efficient LLM/MLLM Reasoningcited by 35
  2. 2025SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Drivingcited by 28
  3. 2025Latent Refinement Decoding: Enhancing Diffusion-Based Language Models by Refining Belief Statescited by 9
  4. 2026White-Balance First, Adjust Later: Cross-Camera Color Constancy via Vision-Language Evaluationcited by 4
  5. 2026Enhancing Numerical Prediction in LLMs via Smooth MMD Alignmentcited by 1
  6. 2026LPCAN: Lightweight Pyramid Cross-Attention Network for Rail Surface Defect Detection Using RGB-D Datacited by 0
  7. 2026Knowledge-Embedded and Hypernetwork-Guided Few-Shot Substation Meter Defect Image Generation Methodcited by 0
  8. 2026MA-LipNet: Multi-Dimensional Attention Networks for Robust Lipreadingcited by 0
  9. 2026The Diffusion Duet: Harmonizing Dual Channels with Wavelet Suppression for Image Separationcited by 0
  10. 2026Detecting Deepfakes with Multivariate Soft Blending and CLIP-based Image-Text Alignmentcited by 0
  11. 2026Efficient Image Super-Resolution with Multi-Scale Spatial Adaptive Attention Networkscited by 0
  12. 2026A Deformable Attention-Based Detection Transformer with Cross-Scale Feature Fusion for Industrial Coil Spring Inspectioncited by 0
14 more
  1. 2026Multi-View Deformable Convolution Meets Visual Mamba for Coronary Artery Segmentationcited by 0
  2. 2026JND-Guided Neural Watermarking with Spatial Transformer Decoding for Screen-Capture Robustnesscited by 0
  3. 2026Transmittance-Guided Structure-Texture Decomposition for Nighttime Image Dehazingcited by 0
  4. 2026Hierarchical Awareness Adapters with Hybrid Pyramid Feature Fusion for Dense Depth Predictioncited by 0
  5. 2026Lightweight Real-Time Rendering Parameter Optimization via XGBoost-Driven Lookup Tablescited by 0
  6. 2026Image Classification via Random Dilated Convolution with Multi-Branch Feature Extraction and Context Excitationcited by 0
  7. 2026Multi-Branch Non-Homogeneous Image Dehazing via Concentration Partitioning and Image Fusioncited by 0
  8. 2026Local Spatiotemporal Convolutional Network for Robust Gait Recognitioncited by 0
  9. 2026Do You Need Text Rectification? Soft Attention Mask Embedding for Rectification-Free Scene Text Spottingcited by 0
  10. 2026DAIN: Dynamic Agent-Based Interaction Network for Efficient and Collaborative Multimodal Reasoningcited by 0
  11. 2026ProWAFT: A ROMA-LPD Instance for Workload-Aware and Dynamic Fault Tolerance in FPGA-Based CNN Acceleratorscited by 0
  12. 2026MissClick: Execution-Aware Adversarial Attacks on Coordinate Generation in GUI Grounding Modelscited by 0
  13. 2026When Tokenization is Secretly Output Supervisioncited by 0
  14. 2026Contour-Guided Spectral Routing for Robust Real-Time Pedestrian Detectioncited by 0