Captioning Daily Activity Images in Early Childhood Education: Benchmark and Algorithm
Not on the map. Why?
Abstract (arXiv)
Image captioning for Early Childhood Education (ECE) is essential for automated activity understanding and educational assessment. However, existing methods face two key challenges. First, the lack of large-scale, domain-specific datasets limits the model's ability to capture fine-grained semantic concepts unique to ECE scenarios, resulting in generic and imprecise descriptions. Second, conventional training paradigms exhibit limitations in enhancing professional object description capability, as supervised learning tends to favor high-frequency expressions, while reinforcement learning may suffer from unstable optimization on difficult samples. To address these limitations, we introduce ECAC, a large-scale benchmark for ECE daily activity image captioning, comprising 256,121 real-world images annotated with expert-level captions and fine-grained labels. ECAC is further equipped with a domain-oriented evaluation protocol, the Teaching Toy Recognition Score (TTS), to explicitly measure professional object naming accuracy. Furthermore, we propose RSRS (Reward-Conditional Switch of Reinforcement Learning and Supervised Fine-Tuning), a hybrid training framework that dynamically alternates between RL and supervised optimization. By rerouting hard samples with zero rewards to supervised fine-tuning, RSRS effectively mitigates advantage collapse and enables stable optimization for fine-grained recognition. Leveraging ECAC and RSRS, we develop KinderMM-Cap-3B, a domain-adapted multimodal large language model. Extensive experiments demonstrate that our model achieves a TTS of 51.06, substantially outperforming state-of-the-art baselines while maintaining superior caption quality, highlighting its potential for specialized educational applications.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 48 references that are in the map. First posted on arXiv 2 Apr 2026.
Builds on
32 of 48 references.
References · 32
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
- 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
- 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
- 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
- 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
20 more
- 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
- 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 466
- 2022AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentationcited by 347
- 2023How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Compositioncited by 222
- 2024VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understandingcited by 116
- 2023Open-domain Visual Entity Recognition: Towards Recognizing Millions of Wikipedia Entitiescited by 112
- 2022MEDFAIR: Benchmarking Fairness for Medical Imagingcited by 66
- 2024Hidden flaws behind expert-level accuracy of multimodal GPT-4 vision in medicinecited by 31
- 2019A Survey on Biomedical Image Captioningcited by 30
- 2024AgroGPT: Efficient Agricultural Vision-Language Model with Expert Tuningcited by 23
- 2023NERIF: GPT-4V for Automatic Scoring of Drawn Modelscited by 13
- 2023Gemini Pro Defeated by GPT-4V: Evidence from Educationcited by 12
- 2024Image Captioning in news report scenariocited by 11
- 2023The Key Artificial Intelligence Technologies in Early Childhood Education: A Reviewcited by 8
- 2024LLaVA-Docent: Instruction Tuning with Multimodal Large Language Model to Support Art Appreciation Educationcited by 8
- 2025VisFinEval: A Scenario-Driven Chinese Multimodal Benchmark for Holistic Financial Understandingcited by 6
- 2025Towards Multimodal Large-Language Models for Parent-Child Interaction: A Focus on Joint Attentioncited by 5
- 2025CFBenchmark-MM: Chinese Financial Assistant Benchmark for Multimodal Large Language Modelcited by 4
- 2024I See You: Teacher Analytics with GPT-4 Vision-Powered Observational Assessmentcited by 1
- 2025DinoCompanion: An Attachment-Theory Informed Multimodal Robot for Emotionally Responsive Child-AI Interactioncited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026SemReward-VL: Semantic Reward-Guided Video-Language Adaptation for Developmental Behavior Assessment
- 2026SpecialEduBench: Benchmarking Vision-Language Models on Knowledge, Skill, and Attitude in Language Intervention for Autistic Children
- 2025Assessing the alignment between infants' visual and linguistic experience using multimodal language models
- 2025Exploring Automated Recognition of Instructional Activity and Discourse from Multimodal Classroom Data
- 2023The Potential of Vision-Language Models for Content Moderation of Children's Videos
- 2025BabyVLM-V2: Toward Developmentally Grounded Pretraining and Benchmarking of Vision Foundation Models
- 2026Structure Over Scale: Learning Visual Reasoning from Pedagogical Video
- 2026Zero-Shot Vision-Language Models for Classroom Engagement Recognition: A Benchmark Study of Prompt Sensitivity and Cross-Dataset Generalization