A copula augmentation of the mean-field approach to automatic differentiation variational inference

A common problem in Bayesian statistics is in evaluating a posterior density when we have access to the corresponding prior and likelihood, but have no practical means of evaluating the corresponding marginalisation. The Markov chain Monte Carlo (MCMC) algorithm is the algorithm that has traditionally been used to find approximations to such difficult-to-compute posterior densities. However, in recent years, variational inference (VI), an alternative approximation algorithm with the same function and objectives as MCMC, has been shown to generally be significantly faster and less computationally expensive than MCMC (and many other such alternative approximation algorithms). Yet VI is generally lacking the development and academic rigor that MCMC and the likes have been subjected to over the years.

In this thesis, an original contribution to a branch of research into VI will be made. Two recently developed variants of VI that attempt to address specific limitations concerning the general computational complexity and accuracy of VI, are automatic differentiation variational inference (ADVI) and copula VI, respectively. In this thesis, a new variant of VI, that will combine elements of the two aforementioned variants, and be referred to in this thesis as "copula ADVI", will be developed, empirically tested and evaluated. The empirical testing and evaluation of copula ADVI will seek to unveil whether combining automatic differentiation and copulas helps to effectively address the specific limitations of VI that have previously been identified, and whether such a combination of developments could offer a promising future potential for VI in general.