# DishSeg24k: A Large-Scale Benchmark for Food Segmentation with Stochastic Expert Decoding

arXiv: https://arxiv.org/abs/2607.23070 · HTML: https://mlsanity.com/paper/2607.23070/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts.
- Builds on 24 of its 41 references that are in the map.
- First posted on arXiv 25 Jul 2026.

## Placed in

- Mixture-of-experts / [Vision mixture-of-experts](https://mlsanity.com/moe/vision-mixture-of-experts/)

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [CCNet: Criss-Cross Attention for Semantic Segmentation](https://mlsanity.com/paper/1811.11721/) (2018)
- [Contextual Markov Decision Processes](https://mlsanity.com/paper/1502.02259/) (2015)
- [On the Representation Collapse of Sparse Mixture of Experts](https://mlsanity.com/paper/2204.09179/) (2022)
- [A Large-Scale Benchmark for Food Image Segmentation](https://mlsanity.com/paper/2105.05409/) (2021)
- [SegMAN: Omni-scale Context Modeling with State Space Models and Local Attention for Semantic Segmentation](https://mlsanity.com/paper/2412.11890/) (2024)
- [Context-Guided Spatial Feature Reconstruction for Efficient Semantic Segmentation](https://mlsanity.com/paper/2405.06228/) (2024)
- [Multi-Scale Representations by Varying Window Attention for Semantic Segmentation](https://mlsanity.com/paper/2404.16573/) (2024)
- [Embedding-Free Transformer with Inference Spatial Reduction for Efficient Semantic Segmentation](https://mlsanity.com/paper/2407.17261/) (2024)
- [Food Ingredients Recognition through Multi-label Learning](https://mlsanity.com/paper/1707.08816/) (2017)
- [MyFood: A Food Segmentation and Classification System to Aid Nutritional Monitoring](https://mlsanity.com/paper/2012.03087/) (2020)
- [SeeDS: Semantic Separable Diffusion Synthesizer for Zero-shot Food Detection](https://mlsanity.com/paper/2310.04689/) (2023)
- [OVFoodSeg: Elevating Open-Vocabulary Food Image Segmentation via Image-Informed Textual Representation](https://mlsanity.com/paper/2404.01409/) (2024)
- [Object Detection in Indian Food Platters using Transfer Learning with YOLOv4](https://mlsanity.com/paper/2205.04841/) (2022)
- [VolTex: Food Volume Estimation using Text-Guided Segmentation and Neural Surface Reconstruction](https://mlsanity.com/paper/2506.02895/) (2025)
- [Feature-Suppressed Contrast for Self-Supervised Food Pre-training](https://mlsanity.com/paper/2308.03272/) (2023)
- [Differential Evolution Integrated Hybrid Deep Learning Model for Object Detection in Pre-made Dishes](https://mlsanity.com/paper/2412.20370/) (2024)

## Abstract (arXiv)

Food segmentation is essential for applications such as intelligent catering, dietary assessment, and recommendation. However, existing benchmarks fail to capture the complexity of real-world dining scenes. The challenges of dense inter-dish overlap, fine-grained class similarity, and extreme long-tail class distributions exceed the fidelity of current datasets. To fill this gap, we introduce DishSeg24k, a large-scale dish-level segmentation benchmark with 24,096 images, 112,281 instances, and 278 fine-grained categories in real-world dining environments. Based on DishSeg24k, we further propose Food Expert-Adaptive Segmentation Transformers (FEAST) to address these challenges. FEAST models query-based decoding as a Markov Decision Process (MDP), where each decoder layer update is treated as a sequential decision step that explores uncertainty along dish boundaries. We further redesign the decoder with a reinforcement learning (RL)-guided Mixture-of-Experts (MoE) module, in which a dual-critic decoupled optimization scheme separates task-oriented query refinement from structure-aware expert routing. This design promotes expert specialization and prevents expert collapse under long-tail category distributions. Finally, extensive experiments on DishSeg24k demonstrate the state-of-the-art performance of FEAST, which outperforms previous methods by +3.21% mIoU, +3.68% mDice, and +4.00% mAcc, respectively. We further validate the effectiveness of FEAST on FoodSeg103. The dataset and code will be publicly released.