# EML-NET:An Expandable Multi-Layer NETwork for Saliency Prediction

arXiv: https://arxiv.org/abs/1805.01047 · HTML: https://mlsanity.com/paper/1805.01047/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 May 2018.

## Built on by (most cited first, 30 of 51)

- [DeepGaze IIE: Calibrated prediction in and out-of-domain for state-of-the-art saliency modeling](https://mlsanity.com/paper/2105.12441/) (2021)
- [In the Eye of Transformer: Global-Local Correlation for Egocentric Gaze Estimation](https://mlsanity.com/paper/2208.04464/) (2022)
- [TranSalNet: Towards perceptually relevant visual saliency prediction](https://mlsanity.com/paper/2110.03593/) (2021)
- [Image-GS: Content-Adaptive Image Representation via 2D Gaussians](https://mlsanity.com/paper/2407.01866/) (2024)
- [Position, Padding and Predictions: A Deeper Look at Position Information in CNNs](https://mlsanity.com/paper/2101.12322/) (2021)
- [TempSAL -- Uncovering Temporal Information for Deep Saliency Prediction](https://mlsanity.com/paper/2301.02315/) (2023)
- [Deep Saliency Prior for Reducing Visual Distraction](https://mlsanity.com/paper/2109.01980/) (2021)
- [GazeXplain: Learning to Predict Natural Language Explanations of Visual Scanpaths](https://mlsanity.com/paper/2408.02788/) (2024)
- [Rectify ViT Shortcut Learning by Visual Saliency](https://mlsanity.com/paper/2206.08567/) (2022)
- [Realistic Saliency Guided Image Enhancement](https://mlsanity.com/paper/2306.06092/) (2023)
- [Good, Cheap, and Fast: Overfitted Image Compression with Wasserstein Distortion](https://mlsanity.com/paper/2412.00505/) (2024)
- [Beyond Average: Individualized Visual Scanpath Prediction](https://mlsanity.com/paper/2404.12235/) (2024)
- [SUM: Saliency Unification through Mamba for Visual Attention Modeling](https://mlsanity.com/paper/2406.17815/) (2024)
- [SalFBNet: Learning Pseudo-Saliency Distribution via Feedback Convolutional Networks](https://mlsanity.com/paper/2112.03731/) (2021)
- [UniAR: A Unified model for predicting human Attention and Responses on visual content](https://mlsanity.com/paper/2312.10175/) (2023)
- [Gaze-directed Vision GNN for Mitigating Shortcut Learning in Medical Image](https://mlsanity.com/paper/2406.14050/) (2024)
- [Brand Visibility in Packaging: A Deep Learning Approach for Logo Detection, Saliency-Map Prediction, and Logo Placement Analysis](https://mlsanity.com/paper/2403.02336/) (2024)
- [SimpSON: Simplifying Photo Cleanup with Single-Click Distracting Object Segmentation Network](https://mlsanity.com/paper/2305.17624/) (2023)
- [Data Augmentation via Latent Diffusion for Saliency Prediction](https://mlsanity.com/paper/2409.07307/) (2024)
- [Multi-task UNet: Jointly Boosting Saliency Prediction and Disease Classification on Chest X-ray Images](https://mlsanity.com/paper/2202.07118/) (2022)
- [Improving saliency models' predictions of the next fixation with humans' intrinsic cost of gaze shifts](https://mlsanity.com/paper/2207.04250/) (2022)
- [CLID: Controlled-Length Image Descriptions with Limited Data](https://mlsanity.com/paper/2211.14835/) (2022)
- [Rethinking Object Saliency Ranking: A Novel Whole-flow Processing Paradigm](https://mlsanity.com/paper/2312.03226/) (2023)
- [GazeFusion: Saliency-Guided Image Generation](https://mlsanity.com/paper/2407.04191/) (2024)
- [Modeling Saliency Dataset Bias](https://mlsanity.com/paper/2505.10169/) (2025)
- [Predicting Visual Attention in Graphic Design Documents](https://mlsanity.com/paper/2407.02439/) (2024)
- [SAMIC: Segment Anything with In-Context Spatial Prompt Engineering](https://mlsanity.com/paper/2412.11998/) (2024)
- [Modeling Object Dissimilarity for Deep Saliency Prediction](https://mlsanity.com/paper/2104.03864/) (2021)
- [Context-empowered Visual Attention Prediction in Pedestrian Scenarios](https://mlsanity.com/paper/2210.16933/) (2022)
- [Enhancing image quality prediction with self-supervised visual masking](https://mlsanity.com/paper/2305.19858/) (2023)

## Abstract (arXiv)

Saliency prediction can benefit from training that involves scene understanding that may be tangential to the central task; this may include understanding places, spatial layout, objects or involve different datasets and their bias. One can combine models, but to do this in a sophisticated manner can be complex, and also result in unwieldy networks or produce competing objectives that are hard to balance. In this paper, we propose a scalable system to leverage multiple powerful deep CNN models to better extract visual features for saliency prediction. Our design differs from previous studies in that the whole system is trained in an almost end-to-end piece-wise fashion. The encoder and decoder components are separately trained to deal with complexity tied to the computational paradigm and required space. Furthermore, the encoder can contain more than one CNN model to extract features, and models can have different architectures or be pre-trained on different datasets. This parallel design yields a better computational paradigm overcoming limits to the variety of information or inference that can be combined at the encoder stage towards deeper networks and a more powerful encoding. Our network can be easily expanded almost without any additional cost, and other pre-trained CNN models can be incorporated availing a wider range of visual knowledge. We denote our expandable multi-layer network as EML-NET and our method achieves the state-of-the-art results on the public saliency benchmarks, SALICON, MIT300 and CAT2000.