An objective Bayesian approach for discrete scenarios

Objective prior distributions represent a fundamental part of Bayesian infer­ence. Although several approaches for continuous parameter spaces have been developed, Bayesian theory lacks of a general method that allows to obtain priors for the discrete case. In the present work we propose a novel idea, based on losses, to derive objec­tive priors for discrete parameter spaces. We objectively measure the worth of each parameter values, and link it to the prior probability by means of the self­information loss function. The worth is measured by taking into consideration the surroundings of each element of the parameter space. Bayes theorem is then re-interpreted, where prior and posterior beliefs are not expressed as probabilities, but as losses. The approach allows to retain meaning from the beginning to the end of the Bayesian updating process. The prior distribution obtained with the above approach is identified as the Villa-Walker prior. We illustrate the approach by applying it to various scenarios. We derive ob­jective priors for five specific models: a population size model, the Hypergeometric and multivariate Hypergeometric models, the Binomial-Beta model, and the Bi­nomial model. We also derive the Villa-Walker prior for the number of degrees of freedom of a t distribution. An important result in this last case, is that the objective prior has to be truncated. We finally apply the idea to discrete scenarios other that parameter spaces: model selection, and variable selection for linear regression models. We show how an objective model prior can be obtained, by applying our approach, on the basis of the importance that each model has with respect to the other ones. We illustrate various cases: nested and non-nested models, models with discrete and continuous supports, uniparameter and multiparameter models. For the variable selection scenario, the prior includes a loss component due to the complexity of each regression model.