Skip to main content

Posterior Collapse in Variational Gradient Origin Networks

Clapham, Peter, Grzes, Marek (2023) Posterior Collapse in Variational Gradient Origin Networks. In: 22nd International Conference on Machine Learning and Applications (ICMLA). . (In press) (KAR id:103129)

Abstract

Posterior collapse is a phenomenon that occurs when the posterior distribution degenerates to the prior, leading to a decline in the quality of latent encodings and generative models. While it is known to occur in Variational autoencoders (VAEs), it is unknown whether it occurs in Variational Gradient Origin Networks (VGONs). The goal of this paper is to compare the posterior collapse of Variational Gradient Origin Networks and Variational Autoencoders. By checking the latent encodings of VGONs against the key posterior collapse metrics, our experiments reveal that VGONs do exhibit posterior collapse both in the decline of the Kullback-Leibler divergence (KLD) and the collapse of individual variables. Furthermore, the results show that VGONs and VAEs have a similar polarized regime, suggesting that the cause of posterior collapse is not specific to the architecture of the model used to find an encoding. These findings support the claim made in previous research that posterior collapse is a general issue that affects a wide range of latent variable models.

Item Type: Conference or workshop item (Paper)
Subjects: Q Science > QA Mathematics (inc Computing science) > QA 76 Software, computer programming, > QA76.87 Neural computers, neural networks
Divisions: Divisions > Division of Computing, Engineering and Mathematical Sciences > School of Computing
Funders: University of Kent (https://ror.org/00xkeyj56)
Depositing User: Marek Grzes
Date Deposited: 05 Oct 2023 13:51 UTC
Last Modified: 06 Oct 2023 10:25 UTC
Resource URI: https://kar.kent.ac.uk/id/eprint/103129 (The current URI for this page, for reference purposes)

University of Kent Author Information

Clapham, Peter.

Creator's ORCID:
CReDIT Contributor Roles:

Grzes, Marek.

Creator's ORCID: https://orcid.org/0000-0003-4901-1539
CReDIT Contributor Roles:
  • Depositors only (login required):

Total unique views for this document in KAR since July 2020. For more details click on the image.