# Conditional Random Field Autoencoders for Unsupervised Structured Prediction

arXiv: https://arxiv.org/abs/1411.1147 · HTML: https://mlsanity.com/paper/1411.1147/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Nov 2014.

## Built on by (most cited first, 8 of 8)

- [Visual Dependency Transformers: Dependency Tree Emerges from Reversed Attention](https://mlsanity.com/paper/2304.03282/) (2023)
- [Latent Topology Induction for Understanding Contextualized Representations](https://mlsanity.com/paper/2206.01512/) (2022)
- [Scaling Structured Inference with Randomization](https://mlsanity.com/paper/2112.03638/) (2021)
- [Learning with Latent Structures in Natural Language Processing: A Survey](https://mlsanity.com/paper/2201.00490/) (2022)
- [Bridging Pre-trained Language Models and Hand-crafted Features for Unsupervised POS Tagging](https://mlsanity.com/paper/2203.10315/) (2022)
- [Modeling Label Correlations for Second-Order Semantic Dependency Parsing with Mean-Field Inference](https://mlsanity.com/paper/2204.03619/) (2022)
- [Masked Part-Of-Speech Model: Does Modeling Long Context Help Unsupervised POS-tagging?](https://mlsanity.com/paper/2206.14969/) (2022)
- [ReMAP: Lossless Residual Neural Lifting for Scalable MAP Inference in Arbitrary-Order Markov Random Fields](https://mlsanity.com/paper/2411.18954/) (2024)

## Abstract (arXiv)

We introduce a framework for unsupervised learning of structured predictors with overlapping, global features. Each input's latent representation is predicted conditional on the observable data using a feature-rich conditional random field. Then a reconstruction of the input is (re)generated, conditional on the latent structure, using models for which maximum likelihood estimation has a closed-form. Our autoencoder formulation enables efficient learning without making unrealistic independence assumptions or restricting the kinds of features that can be used. We illustrate insightful connections to traditional autoencoders, posterior regularization and multi-view learning. We show competitive results with instantiations of the model for two canonical NLP tasks: part-of-speech induction and bitext word alignment, and show that training our model can be substantially more efficient than comparable feature-rich baselines.