# Learning Probabilistic Embeddings for Unsupervised Action Segmentation

arXiv: https://arxiv.org/abs/2607.05263 · HTML: https://mlsanity.com/paper/2607.05263/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 59 references that are in the map.
- First posted on arXiv 6 Jul 2026.

## Builds on (most cited first, 25 of 25)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [HiDDeN: Hiding Data With Deep Networks](https://mlsanity.com/paper/1807.09937/) (2018)
- [The Stable Signature: Rooting Watermarks in Latent Diffusion Models](https://mlsanity.com/paper/2303.15435/) (2023)
- [StegaStamp: Invisible Hyperlinks in Physical Photographs](https://mlsanity.com/paper/1904.05343/) (2019)
- [MBRS : Enhancing Robustness of DNN-based Watermarking by Mini-Batch of Real and Simulated JPEG Compression](https://mlsanity.com/paper/2108.08211/) (2021)
- [Watermarking Images in Self-Supervised Latent Spaces](https://mlsanity.com/paper/2112.09581/) (2021)
- [Towards Blind Watermarking: Combining Invertible and Non-invertible Mechanisms](https://mlsanity.com/paper/2212.12678/) (2022)
- [RoSteALS: Robust Steganography using Autoencoder Latent Space](https://mlsanity.com/paper/2304.03400/) (2023)
- [Watermark Anything with Localized Messages](https://mlsanity.com/paper/2411.07231/) (2024)
- [TrustMark: Universal Watermarking for Arbitrary Resolution Images](https://mlsanity.com/paper/2311.18297/) (2023)
- [Video Seal: Open and Efficient Video Watermarking](https://mlsanity.com/paper/2412.09492/) (2024)
- [DVMark: A Deep Multiscale Framework for Video Watermarking](https://mlsanity.com/paper/2104.12734/) (2021)
- [Proactive Image Manipulation Detection](https://mlsanity.com/paper/2203.15880/) (2022)
- [OmniGuard: Hybrid Manipulation Localization via Augmented Versatile Deep Image Watermarking](https://mlsanity.com/paper/2412.01615/) (2024)
- [V2A-Mark: Versatile Deep Visual-Audio Watermarking for Manipulation Localization and Copyright Protection](https://mlsanity.com/paper/2404.16824/) (2024)
- [InvisMark: Invisible and Robust Watermarking for AI-generated Image Provenance](https://mlsanity.com/paper/2411.07795/) (2024)
- [ProMark: Proactive Diffusion Watermarking for Causal Attribution](https://mlsanity.com/paper/2403.09914/) (2024)
- [OSCAR-Net: Object-centric Scene Graph Attention for Image Attribution](https://mlsanity.com/paper/2108.03541/) (2021)
- [EKILA: Synthetic Media Provenance and Attribution for Generative Art](https://mlsanity.com/paper/2304.04639/) (2023)
- [Mask Image Watermarking](https://mlsanity.com/paper/2504.12739/) (2025)
- [ItoV: Efficiently Adapting Deep Learning-based Image Watermarking to Video Watermarking](https://mlsanity.com/paper/2305.02781/) (2023)
- [ARCHANGEL: Tamper-proofing Video Archives using Temporal Content Hashes on the Blockchain](https://mlsanity.com/paper/1904.12059/) (2019)
- [DECORAIT -- DECentralized Opt-in/out Registry for AI Training](https://mlsanity.com/paper/2309.14400/) (2023)
- [On the Coexistence and Ensembling of Watermarks](https://mlsanity.com/paper/2501.17356/) (2025)
- [Content ARCs: Decentralized Content Rights in the Age of Generative AI](https://mlsanity.com/paper/2503.14519/) (2025)

## Built on by (most cited first, 1 of 1)

- [See the Change, Keep the Flow: Unsupervised Action Segmentation via Spectral-Temporal Representation Learning](https://mlsanity.com/paper/2608.29611/) (2026)

## Abstract (arXiv)

This paper concerns the problem of unsupervised temporal action segmentation for long, untrimmed videos. Recent successful approaches follow a joint representation learning and clustering paradigm, where optimal transport (OT) is adopted to produce pseudo labels for learning frame representations. These approaches alternate between estimating pseudo labels using OT and optimizing the parameters with gradient descent during training, where OT is used for obtaining the final temporal action segmentation. A major limitation of these works is that they learn a deterministic embedding for frame representations. The iterative procedure between learning deterministic embeddings based on pseudo labels and estimating pseudo labels from the learned embedding can thus get quickly stuck in a local optimum. As an alternative, we thus propose to learn a probabilistic embedding for frame representations. The embeddings are modeled by Gaussian distributions and we sample from the distributions before estimating the pseudo labels. We evaluate our approach on several challenging temporal action segmentation datasets and achieve results comparable to, and in some cases, better than the state of the art. Compared to baselines with deterministic embeddings, our approach improves MoF up to 20.7% and F1-score up to 19.0%. Our code is available at https://github.com/derkbreeze/PEOT.