# Weakly-supervised Micro- and Macro-expression Spotting Based on Multi-level Consistency

arXiv: https://arxiv.org/abs/2305.02734 · HTML: https://mlsanity.com/paper/2305.02734/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 88 references that are in the map.
- First posted on arXiv 4 May 2023.

## Builds on (most cited first, 30 of 45)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling](https://mlsanity.com/paper/2102.06183/) (2021)
- [What Makes Training Multi-Modal Classification Networks Hard?](https://mlsanity.com/paper/1905.12681/) (2019)
- [Temporal Action Detection with Structured Segment Networks](https://mlsanity.com/paper/1704.06228/) (2017)
- [The THUMOS Challenge on Action Recognition for Videos "in the Wild"](https://mlsanity.com/paper/1604.06182/) (2016)
- [MIST: Multiple Instance Self-Training Framework for Video Anomaly Detection](https://mlsanity.com/paper/2104.01633/) (2021)
- [UntrimmedNets for Weakly Supervised Action Recognition and Detection](https://mlsanity.com/paper/1703.03329/) (2017)
- [Single Shot Temporal Action Detection](https://mlsanity.com/paper/1710.06236/) (2017)
- [Weakly Supervised Action Localization by Sparse Temporal Pooling Network](https://mlsanity.com/paper/1712.05080/) (2017)
- [W-TALC: Weakly-supervised Temporal Activity Localization and Classification](https://mlsanity.com/paper/1807.10418/) (2018)
- [AutoLoc: Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/1807.08333/) (2018)
- [Revisiting Anchor Mechanisms for Temporal Action Localization](https://mlsanity.com/paper/2008.09837/) (2020)
- [Background Suppression Network for Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/1911.09963/) (2019)
- [SF-Net: Single-Frame Supervision for Temporal Action Localization](https://mlsanity.com/paper/2003.06845/) (2020)
- [Weakly-Supervised Action Localization by Generative Attention Modeling](https://mlsanity.com/paper/2003.12424/) (2020)
- [Weakly-supervised Action Localization with Background Modeling](https://mlsanity.com/paper/1908.06552/) (2019)
- [3C-Net: Category Count and Center Loss for Weakly-Supervised Action Localization](https://mlsanity.com/paper/1908.08216/) (2019)
- [Video-based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms](https://mlsanity.com/paper/2201.12728/) (2022)
- [Weakly-Supervised Action Localization with Expectation-Maximization Multi-Instance Learning](https://mlsanity.com/paper/2004.00163/) (2020)
- [A Hybrid Attention Mechanism for Weakly-Supervised Temporal Action Localization](https://mlsanity.com/paper/2101.00545/) (2021)
- [On the Integration of Optical Flow and Action Recognition](https://mlsanity.com/paper/1712.08416/) (2017)
- [Learning Action Completeness from Points for Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/2108.05029/) (2021)
- [ASM-Loc: Action-aware Segment Modeling for Weakly-Supervised Temporal Action Localization](https://mlsanity.com/paper/2203.15187/) (2022)
- [Cross-modal Consensus Network for Weakly Supervised Temporal Action Localization](https://mlsanity.com/paper/2107.12589/) (2021)
- [Multi-Evidence Filtering and Fusion for Multi-Label Classification, Object Detection and Semantic Segmentation Based on Weakly Supervised Learning](https://mlsanity.com/paper/1802.09129/) (2018)
- [Weakly Supervised Temporal Action Localization via Representative Snippet Knowledge Propagation](https://mlsanity.com/paper/2203.02925/) (2022)

## Built on by (most cited first, 4 of 4)

- [Weak Supervision with Arbitrary Single Frame for Micro- and Macro-expression Spotting](https://mlsanity.com/paper/2403.14240/) (2024)
- [ME-TST+: Micro-expression Analysis via Temporal State Transition with ROI Relationship Awareness](https://mlsanity.com/paper/2508.08082/) (2025)
- [Boosting Micro-Expression Analysis via Prior-Guided Video-Level Regression](https://mlsanity.com/paper/2508.18834/) (2025)
- [PESFormer: Boosting Macro- and Micro-expression Spotting with Direct Timestamp Encoding](https://mlsanity.com/paper/2410.18695/) (2024)

## Abstract (arXiv)

Most micro- and macro-expression spotting methods in untrimmed videos suffer from the burden of video-wise collection and frame-wise annotation. Weakly-supervised expression spotting (WES) based on video-level labels can potentially mitigate the complexity of frame-level annotation while achieving fine-grained frame-level spotting. However, we argue that existing weakly-supervised methods are based on multiple instance learning (MIL) involving inter-modality, inter-sample, and inter-task gaps. The inter-sample gap is primarily from the sample distribution and duration. Therefore, we propose a novel and simple WES framework, MC-WES, using multi-consistency collaborative mechanisms that include modal-level saliency, video-level distribution, label-level duration and segment-level feature consistency strategies to implement fine frame-level spotting with only video-level labels to alleviate the above gaps and merge prior knowledge. The modal-level saliency consistency strategy focuses on capturing key correlations between raw images and optical flow. The video-level distribution consistency strategy utilizes the difference of sparsity in temporal distribution. The label-level duration consistency strategy exploits the difference in the duration of facial muscles. The segment-level feature consistency strategy emphasizes that features under the same labels maintain similarity. Experimental results on three challenging datasets -- CAS(ME)2, CAS(ME)3, and SAMM-LV -- demonstrate that MC-WES is comparable to state-of-the-art fully-supervised methods.