Skip to main content
Kent Academic Repository

FONDUE: an algorithm to find the optimal dimensionality of the latent representations of variational autoencoders

Bonheme, Lisa and Grzes, Marek (2022) FONDUE: an algorithm to find the optimal dimensionality of the latent representations of variational autoencoders. [Preprint] (doi:10.48550/arXiv.2209.12806) (The full text of this publication is not currently available from this repository. You may be able to access a copy if URLs are provided) (KAR id:98787)

The full text of this publication is not currently available from this repository. You may be able to access a copy if URLs are provided.
Official URL:
https://doi.org/10.48550/arXiv.2209.12806

Abstract

When training a variational autoencoder (VAE) on a given dataset, determining the optimal number of latent variables is mostly done by grid search: a costly process in terms of computational time and carbon footprint. In this paper, we explore the intrinsic dimension estimation (IDE) of the data and latent representations learned by VAEs. We show that the discrepancies between the IDE of the mean and sampled representations of a VAE after only a few steps of training reveal the presence of passive variables in the latent space, which, in well-behaved VAEs, indicates a superfluous number of dimensions. Using this property, we propose FONDUE: an algorithm which quickly finds the number of latent dimensions after which the mean and sampled representations start to diverge (i.e., when passive variables are introduced), providing a principled method for selecting the number of latent dimensions for VAEs and autoencoders.

Item Type: Preprint
DOI/Identification number: 10.48550/arXiv.2209.12806
Refereed: No
Other identifier: https://arxiv.org/abs/2209.12806
Name of pre-print platform: arXiv
Additional information: Imported from arXiv
Subjects: Q Science > QA Mathematics (inc Computing science) > QA 76 Software, computer programming,
Divisions: Divisions > Division of Computing, Engineering and Mathematical Sciences > School of Computing
Funders: University of Kent (https://ror.org/00xkeyj56)
Depositing User: Marek Grzes
Date Deposited: 06 Dec 2022 11:04 UTC
Last Modified: 05 Nov 2024 13:04 UTC
Resource URI: https://kar.kent.ac.uk/id/eprint/98787 (The current URI for this page, for reference purposes)

University of Kent Author Information

Bonheme, Lisa.

Creator's ORCID:
CReDIT Contributor Roles:

Grzes, Marek.

Creator's ORCID: https://orcid.org/0000-0003-4901-1539
CReDIT Contributor Roles:
  • Depositors only (login required):

Total unique views for this document in KAR since July 2020. For more details click on the image.