mlsanityData to 14 Oct 2026

ABACUS: Adapting Unified Foundation Model for Bridging Image Count Understanding and Generation

arXiv 2606.23835 ↗First posted 22 Jun 2026cs.CV eess.IVcited by 0

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

We present ABACUS, a unified vision-language model that jointly addresses object counting, crowd counting, referring-expression counting, and count-faithful image generation within a single 3B-parameter model. ABACUS introduces three contributions: density-aware adaptive zooming paired with an objectness map from multi-head self-attention decomposition to spatially ground count predictions; a boundary-aware count policy trained via GRPO with nested local, boundary, and global rewards to eliminate over- and undercounting at crop boundaries; and a cycle-consistent GRPO strategy in which the frozen understanding branch scores generated candidates on count-deviation and aesthetic quality, closing the understanding-generation synergy gap without any external critic or annotation. ABACUS achieves state-of-the-art results across seven benchmarks spanning object counting (FSC-147, CARPK), crowd counting (ShanghaiTech A/B), referring-expression counting (REC-8K), count-faithful generation (CoCoCount, T2I-CompBench, GenEval), and count reasoning (CountQA), surpassing both task-specific specialists and larger generalist models. Project page is at https://mondalanindya.github.io/pages/ABACUS.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 78 of its 98 references that are in the map. First posted on arXiv 22 Jun 2026.

Builds on

78 of 98 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025CountQA: How Well Do MLLMs Count in the Wild?8 Aug
  2. 2024LVLM-COUNT: Enhancing the Counting Ability of Large Vision-Language Models1 Dec
  3. 2024Towards Interpreting Visual Information Processing in Vision-Language Models9 Oct
  4. 2024Revisiting Referring Expression Comprehension Evaluation in the Era of Large Multimodal Models24 Jun
  5. 2023CLIP-Count: Towards Text-Guided Zero-Shot Object Counting12 May
  6. 2023CrowdCLIP: Unsupervised Crowd Counting via Vision-Language Model9 Apr
  7. 2023Teaching CLIP to Count to Ten23 Feb
  8. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation28 Jan

Other references · 70

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2025Qwen2.5-VL Technical Reportcited by 5,587
  6. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  7. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  8. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  9. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  10. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  11. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  12. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
38 more
  1. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  2. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
  3. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 790
  4. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 781
  5. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  6. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 702
  7. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
  8. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  9. 2018nocaps: novel object captioning at scalecited by 444
  10. 2024YOLOv9: Learning What You Want to Learn Using Programmable Gradient Informationcited by 425
  11. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 412
  12. 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 347
  13. 2018CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenescited by 267
  14. 2017Drone-based Object Counting by Spatially Regularized Regional Proposal Networkcited by 156
  15. 2021Learning To Count Everythingcited by 150
  16. 2020Distribution Matching for Crowd Countingcited by 143
  17. 2023Silkie: Preference Distillation for Large Visual Language Modelscited by 130
  18. 2024Training-Free Consistent Text-to-Image Generationcited by 125
  19. 2016A Large Contextual Dataset for Classification, Detection and Counting of Cars with Deep Learningcited by 86
  20. 2018Class-Agnostic Countingcited by 85
  21. 2023V3Det: Vast Vocabulary Visual Detection Datasetcited by 81
  22. 2022CounTR: Transformer-based Generalised Visual Countingcited by 79
  23. 2022An End-to-End Transformer Model for Crowd Localizationcited by 74
  24. 2024Be Yourself: Bounded Attention for Multi-Subject Text-to-Image Generationcited by 69
  25. 2022Zero-Shot Temporal Action Detection via Vision-Language Promptingcited by 65
  26. 2019Precise Detection in Densely Packed Scenescited by 63
  27. 2024Isambard-AI: a leadership class supercomputer optimised specifically for Artificial Intelligencecited by 62
  28. 2024CountGD: Multi-Modal Open-World Countingcited by 59
  29. 2023Zero-shot Object Countingcited by 57
  30. 2018Revisiting Perspective Information for Efficient Crowd Countingcited by 49
  31. 2024DAVE -- A Detect-and-Verify Paradigm for Low-Shot Countingcited by 49
  32. 2024Make It Count: Text-to-Image Generation with an Accurate Number of Objectscited by 47
  33. 2025DC-AE 1.5: Accelerating Diffusion Model Convergence with Structured Latent Spacecited by 47
  34. 2023VLCounter: Text-aware Visual Representation for Zero-Shot Object Countingcited by 45
  35. 2025UniLiP: Adapting CLIP for Unified Multimodal Understanding, Generation and Editingcited by 44
  36. 2023Open-world Text-specified Object Countingcited by 41
  37. 2023Point-Query Quadtree for Crowd Counting, Localization, and Morecited by 40
  38. 2016Counting Everyday Objects in Everyday Scenescited by 34

50 most cited of 70 All 70 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Object Counting Across Modalities: Taxonomies, Benchmarks, Applications, and Open Challenges
  2. 2025Can Current AI Models Count What We Mean, Not What They See? A Benchmark and Systematic Evaluation
  3. 2025Assessing the Visual Enumeration Abilities of Specialized Counting Architectures and Vision-Language Models
  4. 2026HoloCount: A Holistic Visual Counting Benchmark for MLLMs
  5. 2026Counting Circuits: Mechanistic Interpretability of Visual Reasoning in Large Vision-Language Models
  6. 2026Why MLLMs Struggle to Count: Overcoming Individuation and Aggregation Bottlenecks with ConvStack
  7. 2026Count Anything at Any Granularity
  8. 2024AFreeCA: Annotation-Free Counting for All