|
| | | | |
| | | |
| Stat |
Members: 3665
Articles: 2'599'751
Articles rated: 2609
19 January 2025 |
|
| | | | |
|
Article overview
| |
|
|
Posterior Collapse and Latent Variable Non-identifiability | | Yixin Wang
; David M. Blei
; John P. Cunningham
; | | Date: |
2 Jan 2023 | | Abstract: | Variational autoencoders model high-dimensional data by positing
low-dimensional latent variables that are mapped through a flexible
distribution parametrized by a neural network. Unfortunately, variational
autoencoders often suffer from posterior collapse: the posterior of the latent
variables is equal to its prior, rendering the variational autoencoder useless
as a means to produce meaningful representations. Existing approaches to
posterior collapse often attribute it to the use of neural networks or
optimization issues due to variational approximation. In this paper, we
consider posterior collapse as a problem of latent variable
non-identifiability. We prove that the posterior collapses if and only if the
latent variables are non-identifiable in the generative model. This fact
implies that posterior collapse is not a phenomenon specific to the use of
flexible distributions or approximate inference. Rather, it can occur in
classical probabilistic models even with exact inference, which we also
demonstrate. Based on these results, we propose a class of latent-identifiable
variational autoencoders, deep generative models which enforce identifiability
without sacrificing flexibility. This model class resolves the problem of
latent variable non-identifiability by leveraging bijective Brenier maps and
parameterizing them with input convex neural networks, without special
variational inference objectives or optimization tricks. Across synthetic and
real datasets, latent-identifiable variational autoencoders outperform existing
methods in mitigating posterior collapse and providing meaningful
representations of the data. | | Source: | arXiv, 2301.00537 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
|
| |
|
|
|
REGISTER