Bayesian inference for models with infinite-dimensionally generated intractable components

In recent years, great effort has been placed on the development of flexible statistical models, which can capture the rich and diverse structures found in real data. Complex models are often intractable, and they require non trivial techniques for inference. In the Bayesian setting, the most common intractability problem is related with nor­malizing constants which cannot be calculated directly. In this case, MCMC methods are a usefrd tool for posterior simulation of the model parameters, and many ideas have been developed to enable the con­struction of the chains with the desired stationary densities. Fre­quently, ideas applied for posterior simulation from doubly-intractable distributions involve an approximation error; general exact methods are only available for models in which both the data and the param­eters take values in fnite-dimensional spaces.

In the present work we propose a novel idea, based on a series ex­pansion representation of the intractable functions, to enable MCMC simulation for models in which either the data or the parameters are infinite-dimensional. We achieve this by introducing a suitable set of latent variables with unknown and possibly infinite dimension. The MCMC construction is then made for a tractable latent model, from which the density of interest can be recovered through marginaliza­tion.

We illustrate the applicability of the method in various situations. We show that the latent variable construction of the retrospective re­jection sampler commonly known as exact simulation algorithm for diffusions, is a particular case of the latent variable construction we propose. We provide an idea for an alternative exact simulation and inference scheme, through a Markov chain construction. We also present two related nonparamctric mixture models, for time series and regression analysis. Their novelty is in the construction of the mixture weights, which gives them great flexibility but introduces an intractable component generated by the infinite-dimensional parame­ters; we show how our methodology can be applied to enable MCMC inference for these models. We also show how our ideas can be used for inference when the power likelihood for nonparametric mixture models is used; a problem which is of interest in many settings and, to our knowledge, has not been solved without the introduction of some approximation error.

Finally, we discuss the matter of Bayesian consistency for Markov models. Unrelated to the driving theme of the thesis, the problem naturally arises from some of the models studied. We make a first step towards a general result for strong consistency which can be used both for discretely observed diffusions and for the time series model we propose.